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ABSTRACT 


This  thesis  investieated  tne  feasibility  of  voice  data 
entry  for  imagery  intelligence  order  of  battle  reporting. 
Time,  accuracy,  and  efficiency  were  measured  for  20  subjects 
in  an  experiment  physically  simulating  the  use  of  a  light 
table,  optics,  and  an  interactive  computer  system  for 
reporting.  A  Threshold  Technology  Inc.  T600  voice 
recognition  system  was  used  for  a  large,  unstructured 
vocabulary  (255  words)  of  unclassified  Soviet/Warsaw  Pact 
equipment  names,  alphanumerics ,  and  editing  commands.  The 
T600  reccgniton  accuracy  for  this  experiment  was  97. 0% 
without  rejects,  and  95.5%  with  rejects. 

Buffered  voice  and  unbuffered  voice  modes  of  the  T600 
were  evaluated  with  typing:  buffered  voice  was  58%  faster, 
end  unbuffered  voice  41%  faster  than  typing.  Voice  was  else 
found  to  be  as  accurate  as  typing  for  writing  short 
order  of  battle  reports.  Finally,  subjects  preferred 
voice  for  several  criteria  evaluated  before  and  after  the 
experiment . 
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I.  BACKGROUND  LEADING  TO  EXPERIMENTATION 


A.  INTRODUCTION 

This  thesis  investigates  the  potential  application  c? 
automatic  speech  recognition  (ASR)  technology  to  military 
imagery  interpretation  reporting.  It  stems  from  the 
author's  background  in  three  areas:  imagery  interpretation, 
Intelligence  Data  Handling  Systems  (IDES),  and  recent 
exposure  to  the  benefits  of  voice  data  entry  as  an 
alternative  modality  for  interacting  with  machines, 
especially  computers. 

The  need  for  the  thesis  arises  from  two  areas:  the  need 
to  evaluate  and  advance  current  ASR  technology  without  major 
redesign  of  systems;  and  the  need  for  faster,  reliable 
reporting  systems  for  the  Intelligence  community.  Dr.  Wayne 
Lea  and  Dr.  Gary  Poock  called  for  the  evaluation  of  state- 
of-the-art  ASR  equipment,  specifically,  to  evaluate  input 
modalities,  e.g.  voice  versus  typing  [Refs.  1  and  2].  The 
intelligence  community  is  continually  seeking  ways  tc 
improve  performance  of  imagery  sensors  and  exploitation  and 
reporting  systems,  and  is  very  interested  in  ways  of 
reducing  costs  while  improving  the  quality  cf  intelligence 
to  tactical  and  strategic  users. 

The  Soviet  Union  and  the  Warsaw  Pact  countries  are 
expected  to  employ  mass,  mobility,  and  surprise  tactics  in 
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any  future  European  attack  scenario  on  our  North  Atlantic 
Treaty  Organization  (NATO)  Allies.  The  speed  and  range  of 
modern  weaponry  leave  little  or  no  room  for  mistakes  in 
responding  to  crisis  situations.  Decision-making  in  minutes 
or  even  seconds  is  a  requirement  today,  and  is  likely  to  be 
mere  critical  in  the  future  with  the  increased  use  cf 
microelectronic  components  for  sensor  and  weapons  control, 
and  faster,  more  redundant,  survivable,  and  interoperable 
communications  facilities.  National  Commend  Authorities, 
U.S.  Strategic  and  Tactical  Forces,  and  NATO  Theater  Forces 
must  have  accurate,  timely,  and  complete  indications  and 
warning  (I&W)  intelligence  of  the  enemy's  real  intentions 
and  capabilities.  Once  hostilities  begin,  with  today's 
warfighting  technology,  military  commanders  will  require 
near-real-time  (NRT)  combat  information  to  enable  them  tc 
provide  effective  command  and  control  of  their  forces  to 
counter  the  enemy. 

Globally,  intelligence  must  be  available  for 
national  security  decisions  regarding  appropriate 
responses  to  international  terrorism  and  the  unwarranted 
intervention  of  foreign  powers  into  the  affairs  of 
other  nations.  Additionally,  intelligence  is 
required  for  long-range  planning  estimates  to  support  the 
acquisition  of  the  best  possible  mix  of  forces  to 
meet  mission  requirements  in  support  of  basic  U.S. 
policy  and  objectives.  Finally,  intelligence  must 
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continually  support  Strategic  Nuclear  Command  and  Control 
forces  which  rust  alwa/s  he  at  a  sufficient  state  of 
readiness  to  provide  nuclear  deterrence. 


The  following  "oasic  command  and  control  model  in  Figure 
1  was  adapted  frcm  the  work  of  Dr.  Joel  Lawson,  Technical 
Director,  Naval  Electronics  Systems  Command  [Ref.  3].  dt  is 
shown  here  to  illustrate  the  importance  cf  the  intelligence 
process  in  providing  support  to  command  and  control  of 
forces  in  war  and  peace.  Note  that  it  does  little  good  to 
provide  better  sensors  without  also  improving  the  ability  to 
compare  the  information  derived  with  objectives  and 
historical  information  in  conjunction  with  intelligence 
analysis,  inherent  in  the  "compare"  process.  In  the 
reconnaissance  area,  imagery  exploitation  and  reporting 
would  fall  under  the  "compare"  function  of  the  system,  and 
as  such  can  be  a  major  information  "bottleneck"  if  not 
capable  of  effectively  processing  the  sensor  output  to  meet 
the  informatior  needs  of  the  decision-maker. 
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...the  central  problem  of  command  control  Is  producing 
an  up-to-date  geographic  display  of  the  location  of 
"things.”  Besides  purely  the  location  of  things  he  [the 
commander]  needs  to  know  what  [the]  things  are,  what  is 
their  identity,  or  who  do  they  belong  to  and  what  is 
their  status. 


Imagery  is  a  key  source  of  such  information,  and  is  thus  a 
major  contributor  to  the  command  and  control  process. 

Automated  Imagery  interpretation  reporting  systems  have 
been  employed  for  strategic  and  theater  support  for  over  10 
years,  and  new  systems  which  include  exploitation  aids  are 
being  deployed  to  tactical  units  now.  They  have 
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significantly  reduced  the  time  tc  exploit  and  report  all 
types  of  imagery  intelligence.  However,  the  man-machine 
interface  research  and  development  of  these  systems  must 
continue  to  meet  future  challenges  facing  the  intelligence 
community.  Significant  volumes  of  imagery  intelligence  will 
he  available  from  NRT  digital  imagery  sensors  in  the  future, 
and  the  best  possible  man-machine  interface  must  be  sought 
to  effectively  exploit  1RV,  order  of  battle,  targeting,  and 
damage  assessment  intelligence  available  from  imagery. 

Reporting  speed  and  accuracy,  manpower  reducti ons ,and 
increased  throughput  are  worthy  design  goals  for  new  or 
improved  imagery  exploitation  and  reporting  systems.  Voice 
data  entry  is  one  newly  evolving  technology  that  offers 
significant  potential  toward  these  goals.  Dr.  Wayne  lea,  in 
the  introduction  to  his  book  Trends  in  Speech  Recognition, 
1982 ,  said: 

Speech  input  seems  to  offer  a  truly  natural  mode  of 
human-machine  communication  that,  if  attainable  in  a 
cost-effective  way,  would  be  unsurpassed  in  making 
computers  and  other  mechanical  devices  truly  cooperative 
servants  of  mankind,  rather  than  increasing  the  demands 
on  the  human  to  adapt  to  the  machine  [Ref.  4] . 

The  next  section  briefly  overviews  the  functions  of 
imagery  reporting  systems,  provides  some  examples  of 
systems  for  today  and  tomorrow,  and  mentions  some  specific 
requirements  which  lead  to  the  desirability  of  voice 
data  entry  for  imagery  intelligence  reporting. 
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IMAGERY  INTERPRETATION  REPORTING  SYSTEMS 
1.  Functions 


B . 


A  military  imagery  interpretation  system  basically 
functions  to  provide  support  for  first,  second,  and  third 
phase  exploitation  of  multi-sensor  imagery  in  response  to 
tasking  from  parent  or  outside  user  organizations.  These 
phases  represent  three  levels  of  depth  of  imagery  analysis 
in  accordance  with  Defense  Intelligence  Agency  (DIA) 
standard  reporting  procedures,  data  elements,  and 
requirements . 

First  and  second  phase  reports  represent  the  hulk  of 
the  work,  and  are  called  Initial/Supplementary  Fhoto 
Interpretation  Reports  ( IPIRs/SUPIRs ) .  The  IPIR  may  he 
thought  of  as  a  quick,  concise  response  to  time-sensitive 
requirements.  It  is  often  followed  hy  the  SUFIR,  which 
represents  a  mere  detailed  and  thorough  exploitation  effort. 
Third  phase  reporting  is  the  most  detailed,  and  includes 
special  analyses  and  reporting  of  selected  installations  of 
specific  interest  to  users  of  imagery  products. 

Such  reporting  standards  and  systems  grew  out  of 
requirements  forced  hy  large  increases  in  the  volume  of 
available  imagery  during  the  sixties.  During  the  sixties, 
the  volume  of  imagery  exceeded  the  exploitation  capabilities 
hy  a  factor  of  five  to  ten  [Ref.  5],  This  drove  the 
development  of  a  variety  of  imagery  exploitation  and 
reporting  systems  which  came  into  operation  in  the 
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seventies,  and  forced  standards  for  reporting  cn  the  imagery 
intelligence  community  as  a  whole.  These  developments 
permitted  the  sharing  of  imagery  intelligence  via  magnetic 
tape  files  and  bulk  data  transfers  over  communications 
circuits.  It  also  facilitated  the  integration  of  imagery 
intelligence  into  more  general  data  bases,  and  enhanced  the 
corporate  memory"  of  intelligence  units,  since  interpreters 
often  kept  installation  data  in  small  personal  files,  net 
easily  accessed  by  others.  With  better  data  bases, 
exploitation  was  enhanced  and  duplicaticn  of  effort  was 
reduced . 

Today,  imagery  exploitation  systems  are  located 
worldwide  in  support  of  U.S.  military  commanders.  The  focus 
now  is  on  providing  more  integrated  data  bases,  which  are 
optimally  dynamic,  complete,  and  timely.  Kulti-s ource 
imagery  reports  may  be  telecommunica ted  to  and  from  many  of 
the  sites,  and  distributed  tc  users  with  a  valid 
requirement.  Integrated  data  bases  will  afford  producers 
and  users  with  more  responsive,  coordinated  information  in 
time  of  need. 

Imagery  systems  range  from  national  level  to 
tactical  reconnaissance  squadron  level  systems.  They  have 
become  Increasingly  capable  of  supporting  many  tasks 
associated  with  exploitation  and  reporting:  responding  to 
tasking  transmitted  over  telecommunications  networks,' 
managing  interpretation  hardware,  software,  and  data  base 
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resources*  exploiting  the  imagery  to  include  malting 
measurements  on  the  imagery,  correlating  imagery  with  maps, 
composing  reports,  editing  them,  and  ether  support 
functions*  disseminating  reports*  and  automatic  screening 


and  updating  cf  local  imagery  and  multi-sourcp  data  bases. 

2 .  Examples  of  Imagery  Interpretation  Reporting  Systems 
The  DIA  uses  the  Automated  Imagery  Related 

Exploitation  System  (AIRES),  modeled  after  the  PACER  system 
used  by  the  Strategic  Air  Command's  544th  Aerospace 
Reconnaissance  Technical  Wing.  PACER  means  Program  Assisted 
Console  Evaluation  and  Review,  and  consists  of  a  dual 
Honeywell  6080  based  computer  system  and  UNIVAC  1652 


consoles  supporting  the  interpretation  process, 
systems  support  a  wide  variety  of  analyst  functions. 


Both 


A  system  developed  and  installed  in  the  late 


seventies  for  theater  and  tactical  user  support  is  the 
Computer  Assisted  Tactical  Information  System  (CATIS).  This 
system  is  used  by  fixed-site,  imagery  exploitation  units  in 
the  Pacific  Air  Forces  (PACAF),  the  Tactical  Air  Command 
i>TAC),  the  Fleet  Intelligence  Center  for  Europe  and  the 
Atlantic  (FICEURLANT) ,  the  United  States  Air  Forces  in 


Europe  (USAFE),  and  the  training  site  in  Air  Training 
Command  (ATC).  The  imagery  exploitation  support  provided  by 
CATIS  may  be  viewed  in  Figure  2. 


To  provide  highly  mobile  support,  the  Tactical 
Information  Processing  and  Interpretation,  Imagery 


Interpretation  System  (TIPI  IIS)  was  developed,  and  is  being 
deployed  to  Air  Force,  Marine,  and  Army  tactical 
reconnaissance  support  units  worldwide.  The  photo 
interpretation  console  of  the  TIPI  IIS  may  be  viewed  in 
Figure  3,  displaying  a  great  deal  of  modular,  ruggedized 
support  equipment  for  imagery  interpretation  reporting  and 
communications.  This  system  provides  mobile  automation  at 
the  squadron  level,  not  previously  available.  For  example, 
an  interpreter  can  use  a  cursor  in  the  light  table  to  make 
rapid,  accurate  measurements  of  objects  such  as  bridges, 
runways,  and  storage  tanks  and  store  the  answer  on  an 
electronic  scratch  pad  for  later  insertion  into  a  report. 
Reports  are  filled  in  quickly,  using  a  f ill-in-the-blank 
online  report  composer.  They  may  then  be  edited  by  a 
supervisor,  and  distributed  over  secure  commiunications 
links . 

To  perform  side-looking  airborne  radar  (SLAR) 
exploitation  and  reporting  the  TIPI  Manual  Radar 
Reconnaissance  Exploitation  System  (MARRES)  was 
developed,  but  with  a  different  console  (Figure  4).  This 
system  provides  special  readout  of  radar  imagery  that  may  be 
used  in  good  or  bad  weather,  and  is  useful  for  discovering 
enemy  force  movements  in  inclement  weather,  such  as  that 
found  in  Europe.  Unique  man-machine  systems  have  been 
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provided  to  .assist  in  providing  detection  of  changes  in  the 
landscape  or  order  of  battle. 

New  NRT  digital  imagery  reconnaissance  sensors, 
such  as  foward-looking  infrared  imagery  (FLIR),  Synthetic 
Aperature  Radar  vSAR),  or  other  types  of  imagery 
which  can  he  supported  by  sensors  on  tactical  aircraft 
will  result  in  increased  NRT  imagery.  Exploitation 
systems  to  support  the  sensors  must  be  developed  to  provide 
the  additional  support  required.  The  Air  Force  has 
initiated  advanced  developmental  models  to  prepare  for 
such  a  requirement. 

One  system  is  the  Reconna i ssa nee  Reporting 
Facility  developed  to  support  the  Quick  Strike 
Reconnaissance  concept  whereby  the  reporting  facility  would 
receive  NRT  hardcopy  and  softcopy  (digital)  imagery  from 
reconnaissance  aircraft  over  the  forward  edge  of  the  battle 
area.  When  advancing  enemy  forces  posed  themselves  as 
targets  of  opportunity,  imagery  reports  would  notify  the 
strike  center  to  order  nearby  airborne  loitering 
aircraft  to  destroy  the  target.  Figure  5,  top  and 
bottom,  gives  views  of  the,  shelter  developed  to  test 
the  NRT  reporting  concept. 
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Hf'ure  6.  Compass  Preview 
Digital  Kxploi tation  System 
^Courtesy  of  Northrop  Corporation) 


The  P.RF  contains  computers,  communications,  and  both 
hardcopy  and  softcopy  imagery  exploitation  and  reporting 


stations.  Used  during  exploitation  of  a  target-rich 
wartime  environment,  this  facility  would  pose  a 
challenging  work  environment  for  the  best  of  interpreters 
end  supervisors.  Efforts  to  optimize  the  man-computer 
interface  could  only  result  in  improved  responsiveness  and 
greater  system  capability. 

Another  system,  for  strategic  use,  is  the  Compass 
Freview  digital  imagery  exploitation  system  shown  in  Figure 
6.  For  the  first  time,  interpreters  will  be  able  to  view 
stereo  images  without  the  aid  of  a  light  table,  hardcopy 
imagery,  or  a  stereoscope.  The  interpreter  can  use  computer 
support  to  enhance  the  image  to  improve  its  interpretability 
in  terms  of  scale,  contrast,  sharpness,  and  other  image 
qualities.  S imul taneously ,  historical  data  base  information 
and  reporting  formats  are  available  for  reporting  what  is 


seen  on  the  image  and  correlated  with  other  data. 
Measurements  may  also  be  made  using  a  Joystick  and  cursor. 

The  imagery  systems  discussed  represent  a  large  leap 
forward  in  imagery  intelligence  since  the  late  sixties.  The 
results  from  current  systems  such  as  PACER  and  CATIS  are 
encouraging  with  3:1  and  12:1  increases  in  output  as 
compared  to  their  predecessors,  less  d\>plication  of  effort, 
increased  validity  of  reporting,  and  most  importantly, 
better  responsiveness  to  specific  user  questions. 
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Imagery  reporting  systems  are  quite 
sophisticated,  having  incorporated  not  only  state-of- 
the-art  exploitation  techniques,  hut  others  as 
well  from  computer,  communications,  and  other 
intelligence  disciplines.  Significant  skill  and 
training  are  required  to  operate  them  effectively. 
Interpreters  are  not  trained  typists,  and  thus  their  speed 
may  slow  the  reporting  process.  Additionally,  they  may 
nave  an  inherent  fear  of  working-  with  computers. 
Continuing  attention  must  be  given  to  improving  the 


man-machine  interface  to  optimize  the  system  product: 
complete,  accurate,  and  timely  imagery  intelligence.  Though 
not  a  panacea,  voice  data  entry  may  be  part  of  the  solution 
for  improving  the  imagery  interpretation  systems,  by 
improving  man's  interface  with  the  machine,  and  making 
optimal  use  of  man's  skills  as  an  image  analyst. 

3 .  Requirement  for  Voice  Data  Entry 

During  the  author's  recent  assignment  at  the  Armed 
Forces  Air  Intelligence  Training  Center,  he  was  responsible 
for  managing  the  Initial  development  of  the  TIPI  IIS 
Operator  and  Supervisor  Courses.  As  he  observed 
interpreters  training  cn  the  prototype,  it  was  often 
apparent  that  they  were  deficient  in  typing  skills.  It  was 
painfully  obvious  that  the  multi-million  dollar  IIS  would 
not  produce  reports  any  faster  than  the  few  words-per-minute 
of  the  "hunt  and  peck"  typist.  Certainly,  with  practice 
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individuals  may  improve  their  typing  speed  and  accuracy  as 
they  adapt  to  a  system,  but  as  we  have  seen,  the  trend  is 
toward  faster  reporting,  and  somehow  the  problem  cf  date 
entry  must  be  attacked  or  critical  resources  will  be  wasted 
cn  systems  limited  by  the  the  man-in-the-1 oop . 

One  simple  and  effective  way  may  be  to  conduct 
typing  classes  to  improve  interaction  with  the  computer.  In 
fact,  online  routines  for  teaching  better  typing  could  be 
developed  to  improve  the  interpreters'  skills  between 
missions.  Another  way  may  be  to  use  voice  data  entry,  which 
offer?'  a  great  potential  beyond  even  the  fastest  typists  for 
data  entry,  should  be  easier  and  faster  to  train,  and  could 
be  used  in  conjunction  with  typing,  function  keys,  or  a 
variety  of  other  input  modalities. 

C.  AUTOMATIC  SPEECH  RECOGNITION 
1 .  Overview 

Automatic  Speech  Recognition  (ASR)  is  no  longer  a 
dream  of  the  future,  but  a  technology  being  applied  around 
the  world  by  people  who  use  machines,  allowing  effective 
machine  control  and  data  entry  into  computers.  ASR  is  not 
without  problems  or  limitations  however,  and  must  be 
carefully  examined  before  trying  to  apply  it.  Human  factors 
must  be  studied  and  tailored  to  the  application  to  allow  ASR 
to  have  the  appropriate  impact  it  affords.  Failure  to 
attend  to  operator  considerations  such  as  microphone 
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rrounting,  recognition  accuracy,  error  correction,  response 
time  and  delay,  feedback  and  prompting,  stability  of 
reference  data,  and  training  procedures  can  have 
catastrophic  effects  on  system  performance  for  both  the 
voice  system  and  the  system  it  aids  [Ref.  7]  . 

The  ultimate  goal  for  speech  recognition 
science  is  to  develop  "speech  understanding  systems" 
which  give  the  appropriate  response  to  the  user's 
request,  and  do  not  just  recognize  tne  elements  of  speech  or 
words  and  phrases  [Ref.  8].  Admittedly,  the  technology  is 
not  that  far  along,  but  many  applications  do  not  need  or 
cannot  afford  the  ideal  speech  system.  The  question  that 
must  be  asked  now  is  "what  applications  can  be 
accomplished  in  a  more  cost-effective  manner  witn  voice 
recognition  systems  that  are  available  now  or  will  be 
available  within  the  next  few  years?" 

Speech  scientists  have  been  working  on  ASP  for  about 
28  years.  Commercially  available  speech  recognizers  became 
available  in  1972  with  Scope  Electronics,  Inc.  and  Threshold 
Technology  Inc.  delivering  quality  systems  which  achieved 
significant  results  under  a  variety  of  conditions.  In 
general,  recognition  accuracy  scores  from  99.0%  to  99.9% 
accuracy  have  been  achieved  in  laboratory  conditions  of  no 
noise,  adequate  talker  training,  and  consistent  talking 
habits.  Field  testing,  ncvever  has  usually  achieved  results 
in  the  neighborhood  of  97%  recognition  accuracy,  generally 
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as  a  result  cf  high  background  noises  or  speaking  tc  the 
system  in  a  manner  different  than  the  way  the  system  was 
trained  initially. 

All  ASR  systems  fall  into  either  of  two 
categories:  continuous  (connected)  or  isolated  (discrete) 
speech  systems  [Ref.  9].  Continuous  speech  systems  work  on 
the  extraction  of  information  from  strings  of  words 
that  may  be  run  together  in  natural  speech  in  the  form  of 
strings  of  digits,  phrases,  or  sentences.  Is  dated -word 
recognizers  require  that  a  short  minimum-duration  pause 
be  inserted  between  digits,  words  or  phrases  which  must  be 
spoken  within  a  given  period  of  time,  e.g.  two  seconds. 

These  isolated-word  recognizers  are  more  prevalent 
today  as  they  are  less  expensive,  more  accurate,  work  in 
real-time,  ana  are  more  readily  available.  Continuous 
speech  systems,  however,  may  be  available  within  the  next 
few  years  offering  250  word  vocabularies  and  recognition 
in  real-time  at  a  reasonable  price.  Continuous  speech 
systems,  in  the  upper  end  of  the  cost  spectrum,  are 
approximately  $100,000.  Riga  quality  isolated-word  speech 
recognizers  normally  cost  in  the  tens  of  thousands  of 
dollars  today)  however,  a  few  companies  are  also  introducing 
systems  on  the  market  for  a  few  thousand  dollars  that 
can  recognize  vocabularies  of  about  250  words  with 
recognition  accuracies  of  971  or  better,  according  tc 
Dr.Poock,  who  intends  to  compare  such  systems  at  NPS  for 


command  and  control  applications.  At  the  bottom  end 
of  the  cost  spectrum,  hobby  systems  are  currently  available 
for  a  few  hundred  dollars. 

Dr.  Lea,  well  recognized  for  nis  work  in  speecn  science 
at  the  University  of  Southern  California  and  the  Speech 
Communication  Research  Laboratory  said  this  about  the  future 
of  speech  recognition  technology: 


The  next  ten  years  or  more  wouli  seem  to  offer  a  growing 
spectrum  of  available  devices,  ranging  from  very  low 
cost  isolated  word  recognizers,  through  digit  string 
recognizers,  recognizers  of  strictly  formatted  word 
sequences,  task-restricted  speech  understanding  systems, 
and  more  powerful  research  systems  for  continuous  speech 
recognition.  All  such  systems  will  take  advantage  of 
low-cost  miniaturization  hardware  that  puts  speech 
recognizers  within  the  reach  of  most  potential  users... 
User  acceptance  c?  voice  input  will  approach  the 
"matter-of-fact"  attitudes  now  prevalent  with  limited 
keyboard  entry,  even  though  full  versatility  and 
"habitability"  of  input  languages  will  not  have  been 
attained  to  any  major  degree...  Despite  all  these 
advances,  we  will  be  far  from  the  science  fiction  image 
of  fully  versatile  voice  interaction  with  machines,  and 
I  doubt  that  unrestricted  "phonetic  typewriters”  are  a 
part  of  the  next  decade  or  more  of  practical  work  on 
speech  recognition  [Ref  10] . 
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Value  of  Speech  Recognition 
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Speech  systems  offer  the  potential  to  capitalize 
on  the  best  of  man's  communicative  abilities,  give  him 
compatibility  with  unusual  circumstances,  and  help  him  gain 
additional  mobility  and  freedom  in  some  situations  [Ref. 
11].  Speech  is  said  to  be  the  human's  most  natural 
communication  modality.  It  is  familiar,  convenient,  and  can 
be  used  spontaneously  because  the  individual  uses  it  often 
in  all  types  of  situations.  Though  performance  with  voice 
may  degrade  under  situations  of  stress,  it  may  not  degrade 
as  much  as  a  less  learned,  less  frequently  used  skill. 
Since  voice  is  familiar  to  the  user,  it  is  less  difficult  to 
train  him  to  use  the  system.  Additionally,  voice  is  the 
human's  highest-capacity  output  channel,  and  permits 
simultaneous  communications  with  humans  and  machines.  Fcr 
example,  a  speaker  in  a  large  auditorium  or  a  command  center 
can  display  the  next  visual  on  a  large  screen  display  by 
saying  some  key  phrase  or  word  which  has  meaning  to  both 
listener  and  display  system.  To  illustrate,  when  Dr.  Pocck 
recently  briefed  a  group  of  senior  naval  officers  in  the 
Pacific,  he  used  such  key  phrases  as  "Good  Morning 
Admiral..."  to  begin  his  briefing,  and  "here  you  see  the 
(pause-)  SHIPS  ..."  to  convey  briefing  information  and  tell 
the  command  and  control  graphics  display  system  to  present 
the  next  graphic  in  his  presentation  on  the  subject  of  Voice 
Input  for  Command  and  Control.  This  is  just  one 
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illustration  of  the  creative  ways  man  can  use  voice  input  to 
his  advantage. 

Navy  feasibility  studies  sponsored  by  Naval 
electronics  Systems  Command,  and  conducted  by  Dr.  Poock, 
examined  the  potential  for  voice  data  entry  for  commend, 
control,  communications,  and  intelligence.  Two  voice 
recognition  systems  were  Installed  in  late  I960  at  Fleet 
Headquarters,  Commander-in-Chief  of  the  Pacific  (CINCPAC)  in 
Hawaii  to  examine  the  benefits  and  limitations  of  voice 
input  for  operation  of  the  Worldwide  Military  Command  and 
Control  Time-Sharing  System  (WWMCCS  TSS)  and  the  nearby 
Ocean  Surveillance  Intelligence  System  (OSIS).  One 
advantage  of  many  of  the  new  voice  terminals  is  that  they 
are  stand-alone,  intelligent  terminals  with  standard 
ccmmunicat lens  interfaces  and  character  sets  that  can  be 
interfaced  rapidly  with  computers  possessing  those  seme 
generic  interfaces.  Voice  units  may  be  moved  around  easily 
and  installed  as  simply  as  most  ether  modern  RS-232  plug- 
compatible  terminals.  Voice  may  also  be  used  remotely  as 
much  as  £000  feet  from  the  main  computer,  free  from  any 
panel  space,  displays,  or  complex  apparatus. 

The  advantages  of  voice  input  fer 
complementing  the  communicative  abilities  of  man  are 
offset  somewhat  today  since  a  user  cannot  speak  totally 
naturally,  but  must  insert  pauses  in  between  -utterances,  and 
must  use  utterances  within  the  constraints  of  the  voice 
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system's  stored  vocabulary.  This  requires  the  user  tc 
be  very  familiar  with  the  vocabulary  in  use,  rot 
unlike  knowing  the  letters  of  the  alphabet. 

Speech  input  for  machines  is  also  of  value  in 
helping  man  cope  with  unusual  circumstances.  For  example, 
it  can  be  used  in  complete  darkness,  around  obstacles,  by 
the  blind  and  other  handicapped  individuals,  is  unaffected 
by  weightlessness,  and  only  slightly  affected  by  high 
acceleration  and  mechanical  constraints.  On  the  negative 
side,  it  is  cften  sensitive  tc  dialect,  and  also  susceptible 
to  background  noise  and  distortions.  Additionally,  a 
microphone  must  either  be  worn  or  held  in  close  proximity  to 
the  speaker.  And  finally,  a  display  or  synthesized  voice 
feedback  may  be  necessary  for  tasks  requiring  data  entry 
validation. 

The  mobility  possible  with  voice  input  is  one  of  its 
greatest  attributes.  It  enables  operation  of  devices  from  a 
distance  and  from  various  orientations,  permits  simultaneous 
use  of  hands  and  eyes  for  other  tasks,  and  can  even  permit 
the  telephone  to  be  used  as  a  computer  terminal.  Some 
degree  of  privacy  is  lost,  although  users  often  operate  in 
the  laboratory  at  NPS  inconspicuously  running  graphics 
displays  and  other  command  and  control  applications  without 
bothering  other  nearby  terminal  operators. 

The  key  questions  to  keep  in  mind  when  considering 
the  value  of  speech  input  are:  "is  there  an  application  that 


could  be  done  more  cost-effectively  using  vcice  as  a  single 


or  additional  input  modality? .. .and , 


Is  the  current 


technology  adequate  to  provide  the  quality,  naturalness,  and 
speed  that  the  application  of  interest  requires?"  A  brief 
loch  at  the  military's  efforts  in  voice  technology  may  help 
the  reader  to  further  assess  the  value  of  speech  technology 
for  his  own  application. 

3 .  Military  Researcn  and  Applications 

Research  supported  by  the  Advanced  Research  Projects 
Agency  ilARFA),  which  funds  leading-edge  technology,  was  a 
prime  ingredient  contributing  to  the  development  of  voice 
technology.  However,  a  large  number  of  military  projects, 
such  as  the  ARPA  Speech  Understanding  Research,  met  with 
limited  success  as  a  great  deal  of  work  in  acoustic- 


phonetics,  speech  perception. 


linguistics. 


psychcacoustic  equipment  is  still  necessary  to  provide  the 
foundation  for  ASR  to  approach  human  performance  [Ref  12]  . 

Most  of  the  research  in  the  military  has  turned  to 


taking  off-the-shelf  isolated-word  recognizers  and  adapting 
them  to  particular  applications.  Recognition  studies  in  the 
military  have  been  done  for  applications  in  aircraft 
cockpits,  tactical  field  data  entry,  military  training 
systems,  cartography,  command  and  control  of  networks, 
wargames  and  graphics,  keyword  spotting  of  communications 


channels,  emergency  action  message  composition,  and  imagery 
interpretation  tasks  such  as  mensuration  and  reporting.  The 
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applications  most  closely  related  to  this  thesis  are  the 
cartography,  command  and  control  of  displays,  and  imagery 
interpretation  reporting. 

A  significant  amount  of  research  was  performed  for 
the  Defense  tapping  Agency(DKA)  by  contractors  under  the 
program  management  of  the  Air  Scree's  Rome  Air  Development 
Center (RADC ) .  The  Defense  Mapping  Agency  Aerospace  Center 
( DM  AAC )  and  the  Defense  Mapping  Agency  Aerospace  and 
Hydrographic  Center  IDMAHC)  produce  large  volumes  of 
cartographic  products  for  the  military  and  other  users. 
Research  has  been  performed  for  such  applications  as  voice 
data  entry  for  the  processing  of  Digital  Landirass  System 
(ELMS)  data,  preparation  of  Flight  Information  Publications 
(FLIPS)  data,  and  ocean-depth  measurements  for  digitized 
cartographic  applications.  In  these  applications  analysts 
were  performing  tasks  in  an  "eyes  busy,  hands  busy 
environment,"  sometimes  with  stereo  optics  end  or  other 
special  devices.  Voice  was  shown  experimentally  to  be 
faster,  easier,  and  a  less  fatiguing  mode  of  data  entry  than 
the  more  conventional  modes  used  [Refs.  13,  14,  end  15]. 
User  acceptance  and  system  support  can  be  significant 
problems,  as  explained  by  EMAAC  officials  to  the  author 
during  a  recent  visit  to  their  facilities. 

The  NPS  is  currently  performing  voice  data  entry 
research  in  the  area  of  command  and  control  applications. 
In  a  study  by  Focck,  twenty-four  command  and  control 
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students  operated  the  ARP A  network  or  ARPANET,  a  distributed 
network  of  computers  in  the  U.S.  and  Europe,  using  voice  and 
typing  as  a  comparison  between  the  two  modes  [Ref.  16]. 
Voice  was  significantly  faster  and  more  accurate  fcr 
entering  commands  into  the  system.  Additionally,  students 
were  given  an  secondary  transcription  task  to  perform  while 
operating  the  ARPANET.  The  voice  mode  permitted 
substantially  more  data  to  be  transcribed  than  the  typing 
mode.  On  the  other  hand,  McSorley  recently  demonstrated 
that  voice  was  no  faster  than  typing  for  entering  commands 
into  a  vargame.  This  was  due  in  part  to  the  peer  editing 
features  of  the  game,  but  demonstrates  that  voice  is  not  for 
everything  [Ref.  1?] . 

In  the  area  of  imagery  interpretation,  interest  in 
voice  data  entry  is  growing.  RADC  recently  completed  a 
study  which  evaluated  a  voice  recognition  system  knewn  as 
"Talk  and  Type,”  built  by  Threshold  Technology  Inc.,  to 
study  the  application  of  voice  data  entry  to  the  problem  of 
imagery  interpretation  and  intelligence  report  generation 
[Ref.  18].  The  innovation  by  Threshold  required  the  user  to 
type  the  first  letter  cf  the  word  tc  be  recognized.  In  this 
ranner  the  voice  system  restricted  the  size  of  the 
vocabulary  to  be  searched,  thereby  increasing  recognition 
accuracy.  Four  varied  tests  were  performed  looking  at  small 
and  large  vocabularies,  and  especially  tasks  where  the 
subject  was  describing  scenes  the  way  an  interpreter  might 
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describe  <3  bridge  or  a  runway.  The  results  showed  the  Talk 
and  Type  system  tc  be  superior  ever  typing  for  unskilled 
typists . 

Scon  the  new  ground  station  for  the  Tactical 
Recor.r.aissance-1  iTR-1)  aircraft  is  expected  tc  be  built  tc 
provide  exploitation  and  reporting  support  for  the  sensors 
aboard  the  iJ-2  derivative  aircraft  which  is  expected  to 
provide  NRT  reconnaissance  support  to  theater  forces. 
According  to  the  program  manager,  voice  data  entry  is  a 

serious  consideration  for  inclusion  into  the  program. 

D.  SUMMARY 

The  purpose  of  this  thesis  is  to  investigate  the 

potential  application  of  ASR  technology  to  military  imagery 

interpretation.  The  research  responds  to  the  need  for 

rapid,  concise,  valid  information  for  command  and  control 

of  forces  in  peace  and  war.  The  functions  of  the  imagery 

reporting  systems  include  support  for  a  variety  of 

tasks,  especially  composing  reports.  The  specific  focus 

cf  the  thesis  is  to  examine  the  feasibility  of 

writing  order  of  battle  reports  using  a  large  voice 

* 

vocabulary  of  255  words  of  USSR/Warsaw  Pact  military 

equipment  names,  editing  commands,  and  alphanumerics . 

* 

Several  examples  cf  modern  operational  and 
developmental  ima&ery  exploitation  and  reporting  systems 
were  briefly  discussed  which  represent  potential  systems 
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for  application  of  voice  technology.  Incorporation  of  ASR 
technology  could  result  in  improved  capabilities  in  terms 
of  speed,  accuracy,  and  completeness  of  imagery 
reporting.  ASR  technology  makes  optimal  use  of  the  fact 
that  speech  is  man's  most  natural  input  modality, 
while  the  limited  speeds  of  interpreters  typing  may  not 
optimize  advanced  reporting  system  capabilities. 

The  advantages  and  disadvantages  of  speech  were 
presented.  Seme  of  the  value  of  speech  input  awaits 
technological  breakthroughs  and  may  not  be  realized  in  this 


decade . 

The 

military 

is 

not 

waiting 

however , 

and 

seems 

unwilling 

tc 

pay  for 

ail 

the 

basic 

research 

t  0 

push 

continuous  speech  systems.  Instead,  the  military  is  hard  at 
work  wltn  applications  efforts  with  1 iml ted -vocabulary , 
isolated-word,  speaker-dependent  voice  recognition  systems, 
proven  to  be  reliable  and  accurate  for  the  right 
applications,  while  monitoring  and  sometimes  supporting  work 
by  private  contractors,  hopefully  leading  to  practical 
continuous  speech  systems. 

The  objective  of  this  thesis  is  to  support  military 
applications  research  efforts  aimed  at  comparing  input 
modalities,  and  afford  the  intelligence  community  an 
independent  data  point  regarding  the  overall  evaluation 
of  ASR.  This  research  began  independent  of  the  related 
RADC  research,  and  thus  serves  to  underscore  the 
appropriateness  of  voice  data  entry  support  to  the  task. 
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II. 


DESCRIPTION'  01  THE  EXPERIMENT 

A.  CEJECTIYES  ANT  CONSTRAINTS 

The  objective  cf  this  experiment  was  to  determine  if 
state-of-the-art  voice  data  entry  equipment  was  feasible  for 
reporting  imagery-aerived  order  of  battle  (C3)  intelligence 
using  an  interactive  computer  system.  The  experiment  was 
designed  to  determine  if  tnere  was  any  significant 
difference  in  speed,  accuracy,  efficiency,  and  subject 
attitudes  regarding  manual  keyboard  and  voice  data  entry  for 
this  task.  A  large  unclassified  vocabulary  cf  £55  words 
containing  alphanumeri cs ,  commands,  and  representative 
USSR/Warsaw  Pact  equipment  names  was  selected  for  the 
reporting  scenario  (see  Appendix  A).  Based  on  recent 
research,  voice  data  entry  was  expected  to  be  faster,  more 
accurate,  and  preferred  by  subjects  over  manual  keyboard 
data  entry  [Ref.  IS] . 

Accomplishment  of  this  objective  was  constrained  within 
the  research  facilities  of  the  Naval  Postgraduate  School 
; N PS } .  In  the  interest  cf  time  and  money,  the  process  of 
reporting  was  simulated  to  the  maximum  degree  possible 
within  the  constraints  of  available  subjects  and  laboratory 
facilities.  This  simulation,  though  net  ideal,  afforded  an 
effective,  economical  tool  to  accomplish  this  objective. 
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E.  SUBJECTS 

Twenty  subjects  participated  cn  a  volunteer  basis. 
The  group  was  composed  of  IS  military  officers,  and 
two  civilians.  The  military  officers,  representing  the 
Army,  Navy,  Air  Force,  and  Marines  included  17  males  and  1 
female?  tneir  grades  ranged  from  lieutenant  to  Commander 
and  from  Captain  to  Lieutenant  Colonel.  The  civilians 
included  an  employee  cf  the  National  Security  Agency  and 
a  professor  from  the  NPS  Operations  Research  Department. 
The  subjects'  aees  ranged  from  28  to  45,  with  an  average  age 
of  33. 

Seventeen  of  the  subjects  were  enrolled  in  the  Command, 
Control,  and  Communications  (C3)  Curricula  at  NPS,  while  the 
other  two  students  were  from  the  Intelligence  and  Computer 
Science  curriculas.  The  background  of  the  subjects  were 
quite  varied:  special  warfare;  ground  combat;  communications 
maintenance  and  staff;  logistics  staff;  automatic  data 
processing;  training,'  intelligence;  C3  research;  language 
analysis?  electronic  warfare;  Joint  Chiefs  of  Staff;  field 
artillery;  destroyer  group  staff;  combat  development;  C3 
training  and  operations;  end  tactical  C3  flight  operations. 

Nineteen  of  the  subjects  had  experience  with  interactive 
computer  systems  at  NPS.  Eighteen  of  the  subjects  were 
experienced  in  use  of  the  ARPANET,  a  network  of  computer 
systems  available  for  use  by  the  C3  Curricula  and  other 
researchers  at  NFS.  The  two  subjects  without  ARPANET 
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trained  to 


tne  level  necessary  to 


experience  were 
participate  in  the  experiment  with  their  contemporaries, 
since  a  computer  on  the  ARPANET  was  chosen  as  tne  host  for 
the  experiment. 

The  subjects  were,  as  a  whole,  familiar  with  speech 
recognition  as  many  had  either  seen,  used,  or  even  studied 
automatic  speech  recognition.  Eighteen  subjects  had  seen  a 
voice  recognition  system  demonstrated?  12  subjects  had  used 
voice,  primarily  as  subjects  in  one  other  experiment?  and  11 
had  studied  voice  for  a  term  paper,  thesis,  or  work  at  their 
previous  duty  station. 


C.  EQUIPMENT 

1.  Voice  Recognition  System 

A  Threshold  Technology 

Inc.  Model 

T600 

voice 

recognition  system  was  used 

to  represent 

c  omme 

rcia lly 

available,  state-of-the-art  equipment.  The 

T600 

is  a 

speaker-dependent,  isolated-word  recognizer  which 
automatically  recognizes  spoken  words  or  phrases.  These 
words  or  phrases  are  called  utterances  and  must  be  in  a 
range  of  0. 1-2.0  seconds  in  duration  and  must  be  separated 
by  very  short  pauses  of  0.1  second  or  more  [Ref.  19]. 

The  terminal  consists  of  a  threshold  analog  speech 
preprocessor,  an  LSI-11  microcomputer  and  a  digital  RS-232 
input/output  interface,  an  Ann  Arbor  large  character 
display  and  operator  console,  an  operator  console/micropbcne 
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preamplifier,  and  a  tape  cartridge  unit.  The  speech 
preprocessor,  microcomputer,  and  interfacing  elements  are 
contained  in  the  main  terminal  unit  which  was  table  mounted. 
The  remaining  components,  the  display  console,  and  tape  were 
alsc  table  mounted  and  located  with  the  main  terminal  (see 
Figure  7).  A  Shure  SN-10  noise-cancelling  microphone  with 
headset  was  used  for  the  voice  input  tc  the  preamplifier. 

The  T600  combines  analog  and  digital  signal 
processing  technology  to  perform  the  recognition  function. 
The  energy  from  the  spoken  utterance  is  passed  through  19 
bandpass  filters  spanning  the  speech  spectrum.  The  presence 
or  absence  of  each  of  32  acoustic  features  is  determined, 
and  the  appropriate  feature  information  is  extracted  by  a 
combination  of  analog  and  binary  logic.  The  features  are 
either  primary  features  or  phonetic-event  features.  Primary 
features  describe  the  energy  spectrum  by  measuring  local 
maxima  and  the  energy  rate-of-change  relative  to  the 
frequency  of  the  voice  signal.  Phonetic-event  features 
result  from  measurements  corresponding  to  phoneme-like 
events:  vowels,  nasals  and  fricatives.  The  preprocessor 
also  must  determine  the  beginning  and  ending  of  each  word. 


The  T600  has  two  primary  nodes  of  operation: 
training  mode,  and  recognition  rode.  In  the  training  mode, 
the  T620  extracts  a  time-normalized  tenplate  for  each  given 
word.  This  template  consists  of  two  arrays  referred  to  as 
the  most  significant  tit  uvSR;  and  non  extremum  tit  ^NEB). 
The  PSB  indicates  whether  a  particular  feature  has  occurred 
and  the  NEB  Indicates  the  frequency  of  occurrence.  These 
arrays  combine  to  form  the  reference  array  (BAR)  .  When  the 
T600  is  in  recognition  mode,  the  preprocessor  functions  as 
before:  features  are  extracted,  digitized,  and  time 
normalized.  The  resultant  feature  array  (EAR)  is  correlated 
with  the  stored  RARs  in  the  current  active  vocabulary  and 
the  best  correlation  is  selected  as  the  recognized  word. 

As  previously  mentioned,  for  each  utterance  32 
acoustic  features  represented  in  binary  form  and  their  time 
of  occurrence  are  fed  from  the  preprocessor  to  the 
microcomputer  short-term  memory.  The  pattern-matching 
algorithm  subsequently  compares  these  feature  occurrence 
patterns  to  the  stored  reference  patterns  for  the  various 
vocabulary  words  and  determine  the  "best  fit "  for  a  word 
decision.  The  FAR  of  a  test  word  requires  512  bits  of 
information  (32  features  mapped  into  16  time  segments).  The 
RARs  include  1024  bits  per  word  because  of  the  two  part 
arrays . 

When  the  T600  recognizes  a  word  in  its  vocabulary  it 
will  output  a  preprogrammed  string  of  up  to  16  characters 
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associated  with  the  spoken  word.  These  output  strings  can 
be  modified  by  the  user  at  any  time  via  his  ASCII  console, 
which  may  also  be  used  instead  of  voice  to  interact  with  the 
host  computer.  Also  associated  with  each  word  are  training 
prompts  which  are  strings  of  up  to  12  ASCII  characters 
displayed  on  the  CP.T  terminal  to  notify  the  user  of  the  word 
to  be  trained.  The  T 600  used  in  this  experiment  required  10 
training  utterances  per  word. 

Two  types  cf  errors  can  occur  with  the  T600: 
misrecognition  and  rejection.  Misrecognitlon  errors  are 
those  where  an  output  string  is  selected  for  output  that 
does  not  match  the  utterance.  When  the  system  rejects  the 
utterance  as  not  part  of  the  vocabulary  it  signals  the 
operator  with  a  "beep."  These  two  cases  assume  the  word  was 
in  the  vocabulary  and  properly  trained.  Other  errors  are 
called  operator  errors  and  arise  from  mispronunciation, 
using  words  not  in  the  vocabulary,  or  a  variety  of  other 
errors  such  as  speaking  too  fast  or  slow. 

The  T600  used  had  enough  memory  modules  to  maintain 
an  active  working  vocabulary  of  256  utterances. 
Vocabularies  were  input  and  output  using  the  tape  cartridge 
unit.  The  system  reads  and  stores  prompt  and  output  strings 
and  reference  patterns  from  semiconductor  random  access 
memory  onto  rugged,  high-quality  magnetic  tapes  similar  to 
cassette  tape  cartridges.  A  complete  256  word  vocabulary 
may  be  recorded  or  loaded  in  a  few  minutes. 


47 


Two  recognition  nodes  are  available  on  the  T600: 
unbuffered  and  buffered.  In  unbuffered  mode,  the  T600  will 
send  any  output  immediately  to  the  host  computer.  No 
internal  processing  is  performed  on  the  output  strings. 
Eowever,  the  buffered  mode  permits  up  to  126  utterance 
output  strings  to  be  sequentially  stored  in  a  T600  buffer 
for  subsequent  output  as  a  composite  block  of  characters. 
An  ’’erase  function'  may  be  used  to  delete  the  last 
utterance?  an  "interrupt"  function  sends  a  special  user- 
defined  string  to  the  host  and  deletes  the  remainder  of  the 
buffer  contents?  a  "cancel"  function  may  be  used  to  delete 
the  buffer  contents?  and  a  "transmit"  function  will  cause 
the  T600  to  send  the  buffer  contents  to  the’,  host.  The 
utterance  assigned  to  these  functions  may  be  independent  of 
their  function  name. 

2 .  Tachlstoscope 

To  provide  a  simulation  of  the  light  table 
and  optics  portion  of  the  imagery  interpreter's  work 
environment,  the  G-1132  Harvard  Tachlstoscope  was 
selected  from  the  man-machine  laboratory  facilities, 
(see  Figure  6)  The  tachlstoscope  is  an  instrument  that 
can  present  images  cf  material  presented  cn  cards  and, 
as  modified  in  this  experiment,  a  CET  display.  The  card 
images  may  be  presented  by  a  timer  or  changed  at  will  by  the 
subject  using  a  button  switch.  Lighting  may  be  regulated 
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and  multi-images  overlayed.  The  three  primary  uses  of  the 
device  are  studies  on  learning,  perception,  and  attention 
[Ref.  20]. 

However,  in  this  experiment  the  tachi s toscope  was 
used  in  the  following  manner.  The  viewport  seen  in  Figure  S 
simulates  the  optics  through  which  an  interpreter  must  get 
much  of  his/her  data.  The  4"  x  6"  cards  seen  through  it 
simulated  the  imagery  the  interpreter  was  tasked  to  analyze 
and  report.  The  CRT  presented  three  lines  of  data  (40 
characters  each)  providing  visual  feedback  for  voice  data 
entry.  (Note:  Rome  Air  Levelcpment  Center  has  developed  an 
eyepiece  for  a  Bausch  &  Lomb  stereoscope  that  displays  16 
characters  of  data  while  viewing  the  optics;  thus  the  author 
assumed  that  more  data  could  be  displayed  in  the  next  few 
years  to  support  such  visual  feedback,  if  required.) 

The  tachistoscope  viewport  permitted  the  viewing  of 
the  scenario  cards  and  t..e  Ann  Arbor  CRT.  The  card  image 
was  centered  above  the  three  bottom  lines  of  the  large- 
character  CRT.  The  CRT  displayed  the  responses  of  the  T600 
to  the  subject's  utterances,  thereby  providing  visual 
feedback  to  him/her  performing  the  task. 
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3 .  Scenario  Cards  and  Vocabulary 

The  cards  for  the  reporting  scenario  were  used  to 
sirulate  frares  of  imager /.  Because  no  imager/  interpreters 
were  available  in  large  numbers  for  the  experiment  at  NFS, 
the  author  created  the  cards  with  a  ' to  represent  the 
equipment  location  and  annotated  the  "**"  with  the  number 
and  description  of  the  equipment  at  the  point.  All  subjects 
were  provided  with  the  same  information,  i.e.  they  were 
"perfect  imagery  interpreters"  and  any  experience  level  was 
held  constant. 

Figure  10  illustrates  the  format  of  two  sample  cards 
which  had  five  to  eight  objects  and  an  installation  number. 
Zach  card  was  divided  into  four  quadrants  to  simplify  and 
standardize  the  reporting  process  and  scoring. 

Thirty-sii  cards  were  required  for  the  experiment. 
Their  content  was  governed  by  four  criteria:  realism,  an 
even  mix  of  ground,  air,  and  naval  terms,  full  use  of  the 
USSR/ Warsaw  Pact  vocabulary  selected  for  the  experiment,  end 
maintaining  a  balance  in  number  of  characters  among  sets  of 
cards  to  be  used  in  experimental  trials.  The  cards  used  in 
the  experiment  are  shown  in  reduced  size  in  Appendix  B.  The 
larger,  actual  size  cards  seen  in  Figure  10  were  produced 
using  large  print  on  a  Tektronix  4014  terminal  and  its 
associated  thermal  printer. 
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INSTALLATION  0298-T14217 

50  CONFIRMED  ASU-85 ' AIRBORNE  ASSAULT  GUNS 

l 

27  CONFIRMED  ASU-57 ' AIRBORNE  ASSAULT  GUNS 
tt  ! 


**  20  POSSIBLE  M-20  HEAVY  MORTARS 

I 

62  PROBABLE  122-MM  D-30  FIELD  H0UIT2ERS  ** 


48  CONFIRMED  240-MM  BM-24  ROCKET  LAUNCHERS 


INSTALLATION  0199-014197 

1  XX 

16  CONFIRMED  MI-41hOUND  HELICOPTERS 


11  CONFIRMED  MI-12.H0MER  HELICOPTERS 

f 

**  5  PROBABLE  MI-6.H00K  HELICOPTERS 


F  re  1C 
(actual  sice 


Sample  5c er.eric  Carls 
4  X  t  including  border' 


A  USSR/Warsaw  Pact  vocabulary  was  used  because  of 
available  unclassified  source  information  in  large  quantity 
[Refs.  21,  22,  23, and  24).  A  full  vocabulary  cf  255  words 
was  used  containing  the  phonetic  alphabet,  numbers  3-25, 
administrative  alphanumerics ,  special  symbols  and  control 
characters,  and  ground,  air,  and  naval  forces  equipment 
vocabulary.  Appendix  A  contains  a  complete  listing  of  the 
vocabulary  by  number,  training  prompt,  and  output  string. 

The  vocabulary  was  not  structured  in  terms  of 
recognition  sets.  Rather,  the  T62Z  operated  on  the  entire 
vocabulary  each  time  an  utterance  was  spoken. 

4.  Interactive  Computer  System;  ARPANET 

To  provide  an  interactive  text  editing  environment 
for  the  reporting  scenario,  the  facilities  of  the  ARPANET 
were  selected  because  of  their  reliability  and  also  to 
demonstrate  how  reporting  might  be  done  over  a  distributed 
network  of  computers,  rather  than  a  local  host  system.  The 
ARPANET,  now  managed  by  the  Defense  Communications  Agency, 
was  used  by  16  of  the  subjects  during  5  quarters  of  their  C3 
Curricula  prior  to  the  experiment. 

Two  host  computers  were  used:  Information  Sciences 
Institute  Systems  I  and  C  (ISIE  &  ISIC),  located  in  southern 
California.  The  experimental  text  editor  (XED),  phctoscript 
(PHCTO),  directory  linking  (TALK),  file  transfer  protocol 
(FTP),  and  file  archival  (ARCHIVE)  were  the  major  programs 
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used  to  conduct  and  manage  the  experimental  date  end 
interactive  computer  environment.  ISIC  was  the  primary 
system  used,  because  the  "system  load  level"  was  generally 
lower  thereby  offering  a  more  responsive  system.  The  load 
level  was  checked  during  experimentation  to  assure  a 
consistent  response  time  was  available  to  all  subjects. 
Beth  systems  were  supported  by  the  TGPS-22  Operating  System, 
on  Digital  Equipment  Corporation  (DEC)  Model  28  Computers. 

These  computers  were  linked  to  NPS  terminals 
equipped  with  phone  modems  or  acoustic  couplers  via 
the  ARPANET  distributed  communications  facilities. 
These  facilities  include  a  terminal  interface  processor 
vTIP)  at  NPS  connecting  school  terminals  with  IS!  via 


the 

ARPANET. 

The 

author  gained 

access  tc  the 

network 

via 

the  TIP 

and 

selecting  the 

network  computer 

tc  be 

used 

The 

ARPANET  provided 

a  myriad  of  facilities 

supporting  the  administration  cf  tne  experiment,  figure  11 
is  a  map  of  the  ARPANET  adapted  from  the  ARPANET  Information 
Brochure,  1979. 

CRT  terminals  and  the  T6Z0  were  attached  to  the 
ARPANET  via  32£  bps  acoustic  couplers.  A  Lier-Siegler 
ADM  CRT  display  was  situated  near  the  tachi s toscope  to 
provide  keyboard  entry  of  the  CB  data  obtained  from  the 
cards  via  the  viewport  (see  Figure  12).  The  ADM  terminal  on 


r;  be 
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tie  ARPANET  was  used  to  simulate  the  text  editing  facilities 
cf  an  imagery  reporting  system  for  the  order  cf  battle  entry 
portion  of  the  report.  All  keystroke  entries  into  the 
terminal  were  copied  by  a  typescript  program  during  the 
experiment  to  provide  a  record  of  the  subject's  performance. 

A  monitor  station  with  a  hardcopy  Computer  Devices 
Kiniterm  and  an  Alanthus  V-203  CRT  display  were  used  to 
record  and  observe  the  subject's  actions,  whether  by  voice 
or  keyboard  entry  (see  Figure  13).  The  Alanthus  display, 
connected  to  the  T6B0,  provided  the  author  with  a  copy  of 
the  data  being  displayed  to  the  operator  via  the  Ann  Arbor 
display  used  in  the  tachistosc ope  viewport  for  visual 
feedback.  This  was  essential  for  recording,  recognition  and 
rejection  errors  in  the  voice-buffered  mode;  such  errors 
could  not  be  analyzed  from  the  hardcopy  record  if  edited 
from  the  voice  buffer  prior  to  transmission  of  the  buffer 
contents  tc  the  text  editor. 

D.  SUBJECT  PREPARATION 

1 .  T600  Vocabulary  Training 

Prior  to  the  experiment,  subjects  were  individually 
trained  in  the  use  of  the  T6 2Z  to  a  level  of  knowledge  and 
competence  to  allow  them  to  operate  it  to  train  the  large 
vocabulary  of  255  words.  Each  subject  was  briefed  on  the 
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proper  training  of  the  Tfci20,  and  received  a  derronstrat ion 
and  written  instructions  with  tne  training  (see  Appendix  C). 
Cnee  the  subject  had  demonstrated  proficiency  in  operating 
and  training  the  T600,  he/she  was  allowed  tc  proceed 
independently,  with  the  author  remaining  nearby  to  answer 
questions  and  correct  training  -pitfalls.  Once  training  was 
complete,  the  subject  tested  the  vocabulary  by  saying  each 
word  three  times.  Any  words  which  were  misrecognized  or 
rejected  mere  than  once  were  retrained  until  a  good  training 
pattern  was  established.  Most  retraining  was  required 
because  the  subject  fergot  hew  the  word  was  prcncunced  when 
initially  trained. 

The  training  was  normally  accomplished  in  two 
sessions  of  approxima tely  two  hours  each.  Thus  by  the  time 
the  training  was  complete,  the  subject  was  very  familiar 
with  the  T600.  Approximately  four  hours  was  the  average 
time  each  subject  spent  with  the  vocabulary  prior  to 
experimentation.  The  training  patterns  were  stored  on  a 
cassette  tape  for  each  subject  and  retained  by  the  author 
until  experimentation. 
c .  Typing  Test 

A  five  minute  typing  test  was  given  to  each  subject 
to  group  the  subjects  into  FAST"  and  "SLOW"  typing  ability 
groups;  these  groups  were  necessary  for  the  experimental 
design.  The  typing  test  required  only  upper  case  letters 
and  symbols  (Appendix  D),  as  did  the  experiment. 


The  typing  test  was  administered  and  scored 
similarly  to  the  civil  service  test  used  to  screen  clerk- 
typist  applicants  to  determine  their  typing  ability. 
The  typing  tests  were  scored  for  speed  and  accuracy.  A 
raw  score  in  words  per  minute  was  assigned  according  to 
the  number  ,  of  lines  typed.  Credit  was  given  for  all  lines 
typed,  including  any  portion  of  the  last  line  started. 
The  number  of  words  per  minute  was  based  on  an  average  word 
length  of  five  characters.  For  each  mispelled  word,  0.2 
words  per  minute  were  subtracted  from  the  raw  score, 
tnereby  decreasing  the  final  score  to  deduct  for  errors.  For 
exanple,  if  a  subject  had  a  raw  score  of  40  wpm,  but  made  £ 
typing  errors,  the  final  score  would  be  39  wpm. 

Subject  typing  speeds  ranged  uniformly  from  17  to 
56  words  per  minute,  with  an  average  speed  of  43  words  per 
minute.  The  SLOW  typist  scores  ranged  from  17  to  32 
with  an  average  of  25;  FAST  typists  scores  ranged  from  33  to 
£8  with  an  average  of  43. 


3.  Subjective  Questionnaire  and  Data  Sheet 


To  assess  the  attitudes  of  each  subject  before  and 
after  experimentation  regarding  their  assessment  of  voice 
data  entry  versus  typed  data  entry,  a  10  item  subjective 
questionnaire  was  developed  (see  Appendii  I).  The 
questionnaire  asked  for  the  subject's  opinions  regarding  the 
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voice  and  typing  modes  on  concerns  relating  to  usability 
such  as  speed,  accuracy,  flexibility,  training,  and  other 
criteria . 

Subjects  also  completed  a  short  data  sheet  regarding 
age,  previous  job,  background,  next  assignment,  and  voice 
experience.  Appendix  F  contains  tne  data  sheet  format. 

E.  EXPERIMENTAL  PROCEDURE 

As  soon  as  the  subject  completed  the  vocabulary 
training,  he/she  was  scheduled  to  perform,  the  experiment 
which  lasted  between  two  and  four  hours,  depending  on  the 
speed  of  the  subject.  The  experiments  were  conducted  in  the 
NPS  Man-Machine  Lab  at  times  most  convenient  to  the  subject, 
generally  in  the  evening. 

The  subject  was  briefed  concerning  the  general  purpose 
for  the  experiment  and  the  three  major  parts  of  the 
experiment:  typing  mode,  voice-unbuffered  mode,  and  voice- 
buffered  mode  experimental  conditions  (see  Appendix  G). 
Each  experimental  condition  consisted  of  a  practice  card  and 
three  trials.  A  Latin-Square  determined  the  order  of  the 
experimental  conditions  such  that  a  balance  was  maintained 
in  the  numbers  of  people  starting  each  experimental  mode. 
This  balance  was  also  maintained  on  the  second  and  third 
experimental  conditions  for  the  subjects.  In  other  words, 
care  was  taken  that  no  experimental  condition  received  an 


advantage  from  always  being  first,  second,  or  third. 
Subjects  were  assigned  randomly  tc  the  orderings. 

The  subject's  task  for  each  aata  entry  mode  was  to  write 
12  simplistic  on-line  imagery  interpretation  reports  of  the 
USSF/Warsaw  Pact  OB  obtained  from  the  cards  by  looking 
through  the  viewport  of  the  tacni stoscope .  Four  cards  were 
included  per  trial  for  the  three  trials  per  mode. 

Recall  the  sample  cards  in  Figure  le;  they  were  used  for 
typing  (top)  and  voice  (bottom)  modes  respectively,  and 
differed  slightly.  Since  some  utterances  were  actually  two 
or  three  words,  (e.g.  MIG-25  FOXBAT)  and  since  the 
vocabulary  of  equipment  names  were  so  large,  it  was 
unrealistic  to  expect  the  subject  to  recall  which  ones  were 
multiple  words  without  greater  familiarity  with  the 
vocabulary.  A  convention  was  adopted  to  link  such  words 
with  an  underscore  symbol  (_) ,  such  as  MIG-25_F0XBAT ,  to 
remind  the  subject  that  the  name  was  to  be  said  in  a  single 
utterance  vice  two  or  three  utterances.  The  underscore  was 
the  only  distinction  between  the  cards  for  voice  and  typing 
modes . 

The  report  format  is  shown  in  Figure  14.  The  subject 
was  required  to  report  the  installation  number  and  CB 
location  (**)  by  quadrant  in  the  order  shown:  UPPIR  LEFT, 
UPPER  RIGHT,  LOWER  LEFT,  LOWER  RIGHT.  Reports  were  to  be 
separated  by  a  blank  line. 
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INSTALLATION  0298-T14217 
UPPER  LEFT 

27  CONFIRMED  ASU-57  AIRBORNE  ASSAULT  GUNS 
UPPER  RIGHT 

£0  CONFIRMED  ASU-8S  AIRBORNE  ASSAULT  GUNS 

TfiVTTB  TTFT 

20  POSSIBLE  M-20  EEAVY  MORTARS 

48  CONFIRMED  240-MM  BM-24  ROCKET  LAUNCHERS 
LOWER  RIGHT 

62  PROBABLE  122-MM  D-20  FIELD  HOWITZERS 

INSTALLATION  0199-V14197 
UPPER  LEFT 

11  CONFIRMED  MI-12  HOMER  HELICOPTERS 
t  PROBABLE  MI-6  HOOK  HELICOPTERS 
UPPER  RIGHT 

16  CONFIRMED  MI-4  BOUND  HELICOPTERS 
LOWER  LEFT 

19  PROBABLE  MI-24  HIND  HELICOPTERS 
LOWER  RIGET 

21  CONFIRMED  MI-10  HARKE  HELICOPTERS 

Figure  14.  OB  Reporting  Format  Based  on  Cards  in  Figure  10 
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Subjects  were  allowed  short  breaks  between  trials  and 
longer  breaks  between  the  entry  modes  as  they  moved  for 
example  from  the  typing  portion  to  the  voice-unbuffered 
portion  or  vice-versa. 

The  number  of  characters  per  trial  was  balanced  to  a 
very  high  degree  within  10-lb  characters  and  10-15 
utterances  for  all  modes.  The  average  number  of  keystrokes 
per  trial  for  the  typing  mode  was  1170.  The  average  number 
of  utterances  per  trial  for  the  voice-unbuffered  mode  was 
220/trial,  slightly  less  than  the  226/trial  for  voice- 
buffered.  These  keystrokes  and  utterances  did  not  count  any 
editing  keystrokes  or  utterances,  but  included  all  carriage 
returns  required.  To  perform  the  3  modes  x  3  trials,  a 
minimum  of  approximately  3510  keystrokes  and  1344  utterances 
would  be  required,  plus  any  editing. 

Prior  to  beginning  each  experimental  condition  the 
subject  was  briefed  on  the  entry  mode,  reminded  of  the 
editing  features  available  (delete  character,  delete  word, 
delete  line,  and  repeat  line),  and  allowed  to  practice  the 
entry  mode  by  writing  a  report  for  a  practice  card. 

The  experimenter  monitored  the  entire  experiment  at  the 
station  illustrated  in  Figure  13.  The  elapsed  time  to 
complete  each  trial  was  measured  using  a  digital  stopwatch 
and  recorded.  The  Miniterm  provided  a  typescript  for 
analysis  of  the  reports  for  missing  or  extra  information, 
resulting  from  typing  or  voice  recognition  errors.  Extra 


typing  keystrokes  or  voice  utterances  used  for  editing  out 
errors  were  noted  for  subsequent  analysis  for  an  efficiency 
measurement .  The  CHT  display  was  used  for  the  unbuffered 
voice  mode  to  record  the  misrecognitions  end  rejects  since 
they  did  not  appear  on  the  typescript  if  they  were  edited 
prior  to  buffer  transmission. 

At  the  conclusion  of  the  experiment  the  subject 
completed  the  subjective  questionnaire  again.  The  subject 
was  asked  not  to  discuss  the  experiment  with  others. 


F.  DEPENDENT  VARIABLES 

The  following  variables  were  recorded  or  calculated  in 
per  cent  for  each  trial: 

NCC 

Report  Accuracy  ( RA)  -  -  I  100 

NCC  +  OE  +  CE 

where  NCC:  Number  of  Characters  Correct 

OE:  Omission  Errors/missing  data 
CE:  Commission  Errors/extra  data 


NCK/U 

Mode  Efficiency  (ME)  =  - X  100 

NCK/U  +  EX/U  +  EDK/U 

where  NCK/U:  Number  of  Correct 

Key strokes /Utterances  ( Typing /Voice ) 
SK/'U:  Error  Keystrokes/Utterances 
EDK/U:  Editing  Keystrokes/  Utterances 
used  to  recover  errors 
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F.ec cgni lien  Accuracy  (RA) 


NCR 


X  12  2 


NCR  +  Nf! 

where  NCR:  Number  of  Correct  Recognitions 

NM:  Number  of  Misrecognitions 

NCR 

Recognition  Accuracy  (RAR)  =  -  X  100 

with  Rejects  NCR  +  NM  +  NR 

where  NCR:  Number  of  Correct  Recognitions 

NM:  Number  of  Misreccgnitions 
NR:  Number  of  Rejects 

Perhaps  the  most  important  variable  was  the  time  it  took 
for  a  subject  tc  complete  the  trials  in  the  experiment. 
Close  behind,  time  is  accuracy,  since  reports  must  be  valid 
in  addition  to  timely.  Thus  it  is  important  to  look  at 
report  output  in  terms  of  accuracy  as  a  system  product. 
Frequently  experimenters  examine  the  errors  made  with  voice 
and  typing  and  report  the  results  as  percentage  of  error. 
However  in  this  experiment  the  final  test  is  in  the  report 
produced  ...  is  it  accurate?  Next,  how  efficient  is  the 
data  entry  mode?  This  is  also  a  useful  statistic  for 
judging  the  merits  of  each  system.  Accuracy  and  efficiency 
were  basic  measures  of  the  total  system  capability,  i.e.  the 
man  and  the  machine.  Recognition  accuracy  was  a  measure  of 
T600  performance  alone,  with  operator  errors  such  as 
misprcnunc lation  removed.  Two  recognition  accuracy  measures 
were  examined,  but  the  first  is  considered  most  appropriate 
in  this  experiment  since  the  T60£  did  not  output  incorrect 
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data  cut  beeped"  when  it  rejected  what  should  have  been  a 
valid  vocabulary  utterance. 


5.  HYPOTHESIS 

The  following  hypotheses  were  tested: 

1 .  Eypotheses  Regarding  TIME 

a.  E<>  :  There  is  no  difference  in  TIME  to 

complete  reports  between  FAST 
and  SLOW  typists. 

H,  :  E*  false. 

b.  Ho  :  There  is  no  difference  in  TIME  to 

complete  reports  among  the 
THREE  DATA  ENTRY  MODES. 

H|  :  Eo  false. 

c.  Ho  :  There  is  no  difference  in  TIME  to 

complete  reports  among  the 
THREE  TRIALS. 


b.  Ho  :  There  is  no  difference  in  ACCURACY  of 

reports  among  the  THREE  DATA  ENTRY  MODES. 
K /  :  Eo  false. 
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WITHOUT  REJECTS 

a.  H0  :  There  is  no  difference  in  RECOGNITION 

ACCURACY  between  FAST  and  SLOW  typists. 
H/  :  H0  false. 

b.  5$  :  There  is  no  difference  in  RECOGNITION 

ACCURACY  among  the  TWO  VOICE  MODES. 

H i  :  Ho  false. 

c.  E*  :  There  is  no  difference  in  RECOGNITION 

ACCURACY  among  the  THREE  TRIALS. 

Et  :  Eo  false. 
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5.  Hypotheses  Regarding  T603  RECOGNITION  ACCURACY 
WITH  REJECTS 

a.  30  :  There  is  no  difference  in  RECOGNITION 

ACCURACY  WITH  REJECTS  between  FAST  and 
SLOW  typists. 

H i  :  E0  false. 

b.  H*  :  There  is  no  difference  in  RECOGNITION 

ACCURACY  WITH  REJECTS  among  the  TWO 
VOICE  MODES. 

H /  :  H0  false. 

c.  H$  :  There  is  no  difference  in  RECOGNITION 

ACCURACY  WITH  REJECTS  among  the  THREE 
TRIALS. 


fil  :  Ho  false. 

Hypothesis  Regarding  SUBJECT  ATTITUDES 

:  There  is  no  difference  in  SUBJECT 
ATTITUDES  regarding  a  preference  for 
VOICE  DATA  ENTRY  over  TYPED  DATA 


ENTRY  after  the  experiment. 
H  j  ;  Ho  false . 


H.  EXPERIMENTAL  DESIGN 

The  conceptual  design  for  the  experiment  is  illustrated 
in  Figure  15.  This  is  a  three-factor  nested  design  with 
repeated  measures  over  trials.  The  subject  is  nested  within 
only  one  typing  ability  condition.  Recall  that  one-third  of 
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M 


FAST 


TYPING 

ABILITY 

(TA) 


SLOW 


TYPING  VOICE  VOICE 

UNBUFFERED  BUFFERED 

DATA  ENTRY  MODE  (DEM) 


Figure  15 


Conceptual  Design  of  the  Experiment 


another  third  started 


the  subjects  started  typing  first; 
voice-unbuffered  first,  and  another  third  started 
voice-buffered  first. 

An  analysis  of  variance  procedure  was  selected  to  test 
the  hypotheses  for  reporting  times,  accuracy,  and 
efficiency,  and  T 60?  recognition  rates.  A  significance 
level  of  ac  *  3.05  was  used  as  the  experimental  threshold.  A 
sign  test  was  chosen  to  evaluate  the  subjective 
questionnaire  results  at  a  significance  level  of  ac.-  0.10. 

I.  RESULTS 

1 .  Results  for  Reporting  Time 

The  results  for  reporting  time  were  the  most 
significant,  with  an  analysis  of  variance  (ANOVA)  indicating 
SIGNIFICANT  DIFFERENCES  in  the  DATA  ENTRY  f-ODES  and  TRIALS 
ip  <  .0005).  The  mean  reporting  times  in  Table  I  show  the 

average  time  in  minutes  to  complete  each  of  the  reporting 
trials  for  each  of  the  three  data  entry  modes.  Table  II 
displays  the  results  of  the  ANOVA  for  reporting  time,  and 
Figures  16  and  17  illustrate  the  significant  differences. 

On  the  average,  voice-unbuffered  was  41%  faster  and 
voice-buffered  was  56%  faster  than  typed  data  entry.  Thus 
voice  data  entry,  averaging  the  two  modes,  was  50%  faster 
overall  than  typing.  Voice  data  entry  was  faster  because  the 


?Z 


subject 

was 

able  to 

simultaneously 

receive 

informat  ion 

through 

the 

viewport 

while  composing 

the  report.  The 

feedback 

received  on 

the  monitor 

enabled 

immediate 

confirmation 

of  the 

T600  response  to 

his/her 

utterances . 

The  typist,  in  the  conventional  reporting  mode,  was  forced 
to  return  often  to  the  viewport  to  get  additional  items  of 
information,  since  there  was  too  much  to  memorize.  The 
illustrated  differences  may  he  seen  in  Figure  16. 

Learning  over  trials  is  apparent  in  all  three  data 
entry  modes.  Figure  1?  illustrates  the  differences  in  time 
to  complete  the  scenario  by  trials.  No  significant 
differences  were  noted  between  typing  abilities.  All 
subjects  adapted  to  the  reporting  task  well.  The  voice- 
buffered  mode  was  the  most  natural  for  subjects  to  use, 
since  they  could  simply  speak  the  report  into  the  system, 
and  make  corrections  most  easily.  Thus  they  learned  to  use 
it  quickly,  and  improved  slightly  thereafter.  The  voice- 
unbuffered  and  typing  modes,  with  more  room  for  improvement, 
showed  more  learning  as  the  subjects  adapted  to  the 
reporting  scenario. 

No  significant  difference  was  apparent  between  fast 
and  slow  typists  for  this  experiment.  This  was  primarily 
because  the  amount  of  information  that  the  subject  could  get 
from  the  tachistoscope  was  limited  to  the  amount  he/she 
could  memorize  when  moving  back  and  forth  to  the  manual 
keyboard . 
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TABLE  I 


1 


* 


MEAN  REPORTING  TIME  (MINUTES) 


TYPING 


VOICE 

UNBUFFERED 


VOICE 

BUFFERED 


FAST  TYPISTS 

Trial  1 

16.2 

11.6 

10.5 

Trial  2 

13.6 

10.5 

10.1 

Trial  3 

13.2 

9.6 

9.1 

— - - 

- —  — 

— — — 

All  Trials 

14.3 

10.6 

9.9 

SLOW  TYPISTS 

Trial  1 

IS  .0 

12.7 

10.0 

Trial  2 

16.5 

10.6 

9.8 

Trial  3 

15.6 

10.5 

9.2 

—  — — - 

— — — — 

All  Trials 

16.7 

11.3 

9.7 

ALL  SUBJECTS 

Trial  1 

17.1 

12.2 

10.3 

Trial  2 

15.1 

10.7 

10.0 

Trial  3 

14.4 

10.1 

9.2 

—  — — 

— 

All  Trials 

15.5 

11.0 

9.8 

For  the 

following 

analysis  of 

variance  se' 

abbeviations  are 

used  for 

the  sake  of 

brevity.  1 

meaning  is  expanded  below: 


SS:  Sum  of  Squares 
af:  degrees  of  freedom 
MS:  Mean  Stuare 
F:  F  Ratio 

p:  significance  level 
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TABLE  II 


ANALYSIS  CF  VARIANCE  FCR  REPORTING  TIME  (SECONDS) 


SOURCE 

SS 

df 

MS 

F 

P 

BETWEEN  SUBJECTS 

:  3, 588,801. 60 

19 

Typing  Ability 
(TA) 

149,472.05 

1 

149,492.05 

0.78 

NS 

Error 

3,439,329.61 

18 

191,073.87 

WITHIN  SUBJECTS: 

6,588,801.20 

160 

Data  Entry 
l^cde  (DEM) 

3,969,141.28 

2 

1,984,570.64 

61.61 

** 

TA  X  DEM 

167,215.63 

2 

93,607.82 

2.91 

NS 

Error (1 ) 

1,159,579.54 

36 

32,210.54 

Trials  (Tr) 

424,888.41 

2 

212,444.21 

33.22 

** 

TA  x  Tr 

2766.70 

2 

1,383.35 

0.22 

NS 

Error(2 ) 

230,255.50 

36 

6,395.99 

DEM  x  Tr 

66, 396.02 

4 

16,599.01 

2.28 

NS 

TA  i  DEM  x  Tr 

17,872.27 

4 

4,468. e7 

0.61 

NS 

Error (3 } 

525,207.79 

72 

7,294.55 

TOTAL 

10,172,124.80 

179 

**  F  <  0.0005 

r  NS:  NOT  SIGNIFICANT  for  p  <  0.05  ] 
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TYPING 


VOICE-UNBUFFERED  VOICE-BUFFERED 


DATA  ENTRY  MODES  (DEM) 

Figure  16.  Mean  Reporting  Time  by  Data  Entry  Mode 


77 


The  results  for  reporting  accuracy  are  shown  in 
Tables  III  and  IV.  The  analysis  of  variance  for  the 
arcsin-transformed  efficiency  data  revealed  NO  SIGNIFICANT 
DIFFERENCES  in  AIL  CONDITIONS  investigated.  The  subjects, 
whether  fast  or  slow  typists,  did  near  perfect  reporting  in 
each  mode,  over  all  trials.  The  reporting  accuracy  was 
expected  to  be  high,  but  exceeded  the  author's  expectations. 
An  average  of  99.5%  accuracy  was  achieved  for  the 
experiment . 

Subjects  were  told  to  go  as  fast  as  possible,  while 
maintaining  accurate  reporting.  Post  errors  were  errors  of 
omission,  where  a  letter  or  word  was  missing  from  a  report. 
Even  greater  speeds  could  be  expected,  especially  from 
voice,  in  situations  where  more  errors  could  be  tolerated. 
But  in  the  case  of  imagery  reporting,  accuracy  was  deemed 
essential,  even  though  operationally  reports  are  normally 
edited  before  being  sent  cut  to  the  agencies. 

TAELE  III 

MEAN  REPORTING  ACCURACY  (%) 

VOICE  VOICE 


TYPING 

UNBUFFERED 

BUFFERED 

FAST  TYPISTS 

99.8 

99.6 

99.7 

SLOW  TYPISTS 

99.8 

99.4 

99.6 

ALL  SUBJECTS 

99.5 

99.5 

99.6 

TABLE  IV 

ANALYSIS  Of  VARIANCE 

FOR  ARCS  IN-TRANS FOR MID  REPORTING  ACCURACY 
Y  =  2  *  ARCSIN  [S«RT( ACCURACY  %)] 


SOURCE 

SS 

df 

MS 

F 

? 

BETWEEN  SUBJECTS: 

3.786 

19 

Tyring  Ability 
(TA) 

0 . 004 

1 

0.004 

0  .02 

NS 

Error 

3.734 

18 

0.210 

WITHIN  SUBJECTS: 

24.030 

160 

Data  Entry 

Mode  (DEM) 

0.346 

2 

0.173 

1.16 

NS 

TA  i  DEM 

0.407 

2 

0.204 

1 .40 

NS 

Error(l ) 

5.262 

36 

0.146 

Trials  (Tr) 

0.352 

2 

0.176 

1.18 

NS 

TA  x  Tr 

0.202 

r-~ 

c 

0.101 

0  .68 

NS 

Error(2 ) 

5.362 

36 

0.149 

DEM  x  Tr 

0.395 

4 

0.099 

0.64 

NS 

TA  x  DEM  x  Tr 

e.326 

4 

0 . 082 

0.53 

NS 

Error (3 ) 

11.078 

72 

0.154 

TOTAL 

27.518 

179 

[  NS:  NOT  SIGNIFICANT  for  p 
Note:  Arcsin  transform  above  normalizes 

<  0.05  3 
the  per 

cent  data 

• 

79 


Z .  Results  for  Reporting  Efficiency 

The  results  fcr  reporting  efficiency  are  shewn  in 
Tables  V  and  VI.  The  analysis  of  variance  indicated 
SIGNIFICANT  DIFFERENCES  between  the  DATA  ENTRY  ^ODES  . 
Figure  IS  shows  the  differences  with  typing  being  the  most 
efficient  at  95%,  voice-buffered  next  with  an  efficiency  of 
£5%,  and  finally  voice-unbuffered  with  an  efficiency  of  60%. 

The  author  attributes  the  efficiency  difference,  in 
part,  to  the  level  of  experience  with  the  mode.  The  reader 
may  recall  that  the  subjects  had,  in  general,  extensive 
keyboard  experience  during  five  quarters  at  NPS .  In 
comparison  with  typing,  the  subjects  had  very  little 
experience  with  voice.  It  is  expected  that  if  subjects  were 
more  skilled  and  efficient  in  the  use  of  voice  data  entry, 
the  time  advantages  reported  earlier  would  be  even  more 
dramatic.  Voice-buffered  was  more  efficient  than  voice- 
unbuffered  because  the  subject  could  edit  out  an  entire 
incorrect  utterance,  vice  deleting  it  by  voice  a  word  at  a 
time  in  the  unbuffered  mode. 


S0 


MAN  REPORTING  EFFICIENCY  (%) 

VOICE 


TYPING 

UNBUFFERED 

BAST  TYPISTS 

Trial  1 

93.6 

77.2 

Trial  2 

95.1 

80.5 

Trial  3 

93.6 

61.6 

All  Trials 

94.2 

79.6 

SLOW  TYPISTS 

Trial  1 

94.4 

ee  .0 

Trial  2 

95.8 

64.4 

Trial  3 

96.7 

76.9 

All  Trials 

95.6 

60.4 

ALL  SUBJECTS 

Trial  1 

94.0 

76.6 

Trial  2 

95.4 

62,5 

Trial  3 

95.3 

79.3 

All  Trials 

94.9 

60.1 

VOICE 

BUFFERED 


83. 

es. 

83. 

84. 


85. 

84. 

68. 

86.4 


e4.9 

85.0 

65.8 

85.2 


i 
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TABLE  VI 

ANALYSIS  OF  VARIANCE 

FOR  ARCS  IN -TRANSFORMED  REPORTING  EFFICIENCY 
Y  =  2  *  ARCS  IN  [SQRT (EFFICIENCY  %)] 


SOURCE 

SS 

df 

MS 

y 

P 

BETWEEN  SUBJECTS: 

3.059 

19 

Typing  Ability 
(TA) 

0.124 

1 

0.124 

0  .82 

NS 

Error 

2.925 

18 

0.162 

WITHIN  SUBJECTS: 

13.669 

160 

Data  Entry 

Mode  (BEM ) 

7.102 

2 

2.551 

44.95 

## 

TA  x  DEM 

0.023 

2 

0.en 

0.14 

NS 

Error ( 1 ) 

2 .660 

36 

0.079 

Trials  (Tr) 

0.170 

2 

0 . 065 

3.54 

NS 

TA  x  Tr 

0  .020 

2 

0.010 

0  .42 

NS 

Error(2 ) 

e.66e 

36 

0.860 

DIM  x  Tr 

0.157 

4 

0.042 

1 .40 

NS 

TA  x  DEM  x  Tr 

0.201 

4 

0.075 

2.50 

NS 

Errcr(3 ) 

2.166 

72 

0.030 

TOTAL 

16.746 

179 

p  <  0.001 

[  NS: 

NOT  SIGNI 

FICANT  for 

p  <  0.05  ] 
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TYPING 


VOICE-UNBUFFERED  VOICE-BUFFERED 


DATA  ENTRY  MODES  (DEM) 

Figure  IS.  Mean  Reporting  Efficiency  by  Data  Entry  Mode 


4 .  Results  for  T500  Recognition  Accuracy 


The  results  for  the  T600  Recognition  Accuracy  are 
shown  in  Tables  VII,  VIII,  IX,  and  X.  Analysis  of  variance 
cf  the  results  revealed  NO  SIGNIFICANT  DIFFERENCES  for  ALL 
CONDITIONS  considered.  Thus  the  T600  recognized  all 
subjects  equally  well  during  all  trials  of  the  experiment. 
The  T60Z  recognition  accuracy  was  97.0%  overall  if  an  error 
is  defined  as  a  mi sreccgnit ion  only.  If  rejects  are 
included,  the  recognition  accuracy  drops  to  95.5%  as  an 
overall  average. 

These  results  are  based  on  an  average  of  1519 
utterances  per  subject  giving  30,380  utterances  for  the 
entire  experiment  using  20  subjects.  This  number  includes 
the  utterances  required,  plus  misrecogni tions  and  reject 
utterances,  and  finally  the  editing  utterances  used  to 
correct  errors.  A  list  of  misrecognitions  and  rejects  is 
contained  in  Appendix  H. 

The  author  had  expected  the  recognition 
accuracy  to  get  worse  in  later  trials  from  fatigue  or 
frustration,  since  the  experiment  was  two  to  four  hours  in 
length.  One  procedure  that  may  have  helped  was  to 
allow  subjects  to,  upon  their  request,  retrain  troublesome 
words  during  the  course  cf  the  experiment.  The  time  tc 
retrain  was  counted  against  the  trial  time  to  account  for 
realistic  retraining  that  would  take  place  on  the  jot. 


TABLE  VII 

KSAN  T6Ze  RECOGNITION  ACCURACY  (%) 
WITHOUT  REJECTS 


VOICE 

VOICE 

UNBUFFERED 

BUFFERED 

EAST  TYPISTS 

97.0 

97.1 

SLOW  TYPISTS 

97.0 

96.9 

ALL  SUBJECTS 

97.0 

97.0 

TABLE  VIII 

MEAN  T600  RECOGNITION  ACCURACY  [%) 
WITH  REJECTS 


VOICE 

UNBUFFERED 

VOICE 

BUFFERED 

FAST 

TYPISTS 

95.8 

95.4 

SLOW 

TYPISTS 

95.2 

95.4 

ALL 

SUBJECTS 

95.5 

95.4 

TABLE  IX 


ANALYSIS  OF  VARIANCE 

ARCS  IN-TRANSFORMED  T602  RECOGNITION  ACCURACY 

WITHOUT  REJECTS 

Y  =  2  *  ARCSIN  [SORT (ACCURACY  %)] 


SOURCE 

SS 

df 

MS 

F 

P 

BETWEEN  SUBJECTS: 

0.S64 

19 

Typing  Ability 
(TA) 

0.001 

1 

0.001 

0.02 

NS 

Error 

0.863 

18 

0.048 

WITHIN  SUEJECTS: 

1.033 

100 

Data  Entry 

Mode  (DEM/ 

0.000 

'  1 

0.000 

0.00 

NS 

TA  i  DEM 

0.009 

1 

0.009 

0.69 

NS 

Error ( 1 ) 

0.231 

18 

0.013 

Trials  (Tr) 

0  .009 

2 

0.005 

0.63 

NS 

TA  x  Tr 

0.037 

2 

0.019 

2.38 

NS 

Error(2) 

0  .261 

36 

0.008 

DEM  x  Tr 

0.053 

2 

0.027 

2.45 

NS 

TA  x  DEM  i  Tr 

0.032 

c 

0.016 

1.45 

NS 

Error(3 ) 

0.381 

36 

0.011 

TOTAL 

1.897 

119 

[  NS:  NOT  SIGNIFICANT  for  p  <  0.05  ] 
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TABLE  X 

ANALYSIS  Cl  VARIANCE 


ARCS  IN-TRANSFORMED  RECOGNITION  ACCURACY 
1* I TH  REJECTS 

Y  =  2  *  ARCSIN  [SORT (ACCURACY  %)] 


SCURCE 

SS 

df 

MS 

F 

P 

BETWEEN  SUBJECTS: 

0.926 

19 

Tyring  Ability 
(TA) 

0.000 

1 

0.000 

0.0e 

NS 

Error 

0.926 

18 

0.051 

WITHIN  SUBJECTS: 

1.106 

100 

Data  Entry 

Mode  (DEM) 

0.000 

1 

0.00e 

0.00 

NS 

TA  x  DEM 

0.004 

1 

0 . 004 

0.33 

NS 

Error(l ) 

0.224 

18 

0.012 

Trials  (Tr) 

0.001 

2 

0 . 000 

0.00 

NS 

TA  x  Tr 

0.034 

2 

0.017 

2.43 

NS 

Error (2 ) 

0.258 

36 

0.007 

DEM  x  Tr 

0  .046 

2 

0.023 

1.64 

NS 

TA  x  DEM  x  Tr 

0.018 

2 

0.009 

0  .64 

NS 

Srror(3 ) 

0.521 

36 

0.014 

TOTAL 

1.977 

119 

[  NS:  NOT  SIGNIFICANT  for  p  <  0.05  ] 
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During  the  experiment  the  author  observed  that 
subjects  occasionally  became  frustrated  when  the 
T600  was  mlsrecognizing  their  utterances.  This 
frustration  appeared  to,  at  times,  generate  a  lack  of 
confidence  in  the  T €00,  along  with  a  change  in  the 
overall  pitch,  rate,  and  inflection  of  the  voice. 
The  frustration  seemed  more  prevalent  in  the 
unbuffered  than  the  buffered  mode.  For  this  reason,  the 
T600  buffered  mode  was  expected  to  have  a  better 
recognition  rate,  since  it  was  faster  and  somewhat  easier 
to  use.  However  the  results  indicate  there  is  no 
difference  in  the  recognition  rate.  One  explanation  is  that 
subjects  went  faster  in  the  buffered  mode  since  they  could 
correct  the  misrecogni ti cn$  more  easily.  With  the 
consequence  of  a  mi srecognition  reduced,  they  were  less 
afraid  to  make  mistakes. 

5 .  Results  for  Subject  Attitudes 

The  scores  from  the  subjective  questionnaire  given 
before  and  after  the  experiment  were  tested  for  any  general 
change  in  opinion  regarding  voice  versus  typed  data  entry. 
These  scores  were  evaluated  using  a  two-tailed  nonparamet ric 
sign  test,  ac  =  0.10.  A  significant  shift  in  favor  of  voice 
data  entry  over  typing  occured  for  half  of  the  criteria 
covered  by  the  questionnaire.  No  significant  shifts  toward 
typing  resulted  from  the  analysis.  Appendix  I  contains  the 
results  of  the  pre/pcst  questionnaire. 
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Summarizing  the  results,  subjects  either  were 
neutral  or  favored  voice  before  and  after  the  experiment. 
After  the  experiment,  they  preferred  voice  even  mere  for 
ease  of  use,  speed,  flexibility,  intermittent  use,  and 
finally  ease  of  learning  to  use  as  an  input  modality.  They 
continued  to  believe  that  voice  was  a  more  accurate, 
sustaining,  relaxed  man-machine  interface  fer  on-line 
reporting  of  critical,  time-sensitive  information  such  as 
intelligence  obtained  in  a  high-pressure  work  environment. 

The  subjects'  positive  attitudes  about  voice  arise 
from  their  fresh  experience  and  observations  of  speech 
recognition  equipment  in  the  C3  Lab  at  NPS,  where  it  is  used 
with  the  Wargame  Effectiveness  Simulator  (WES)  with  graphics 
and  other  ARPANET  and  laboratory  facilities  to  demonstrate 
its  potential  for  command,  control,  and  ccmmunica tiens 
applications . 
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III.  DISCUSSION 


A.  GENERAL 

This  thesis  investigated  tne  potential  application  of 
automatic  speech  recognition  technology  to  military  imagery 
interpretation  reporting.  Only  the  order  of  battle  portion 
of  reporting  was  investigated  because  of  limited  time  and 
resources.  The  overall  results  of  the  experiment  are 
supportive  of  the  application  of  voice  data  entry  for 
imagery  interpretation  reporting  systems.  Voi ce-buff ered 
mode  was  58%  faster  than  typing,  while  voice-unbuffered  was 
41%  faster.  On  the  average,  voice  was  50%  faster  than 
typing . 

Voice  was  faster  because  it  allowed  the  operator  to  view 
the  image  while  reporting.  This  experiment  modeled 
conventional  imagery  reporting  systems  where  a  light  table 
is  located  next  to  a  computer  console.  The  operator 
must  move  back  and  forth  between  the  light  table  and  the 
console,  or  two  operators  work  together,  with  one 
interpreting  the  imagery,  and  the  other  writing  the  report 
via  the  console.  For  these  situations,  it  appears  voice 
data  entry  would  significantly  improve  reporting  speeds 
and/or  require  only  one  person  per  station  to  perform  the 
task.  For  newer  systems  with  the  keyboard  and  function  keys 
built  into  a  computer  console  with  a  light  table  or  digital 
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display,  voice  nay  not  have  as  significant  an  impact  for 
improving  reporting  speeds. 

Eoth  voice  and  typing  were  very  accurate  for 
the  experimental  task,  with  no  significant  difference 
between  mcdes  and  an  overall  accuracy  of  99.5%.  It  is 
interesting  to  note  these  speeds  and  accuracies  were 
obtained  even  though  subjects  were  less  efficient  with 
either  mode  of  voice.  Voice-unbuffered  had  60.1% 
efficiency,  voice-buffered  had  65%  efficiency,  and 
typing  had  95%  efficiency.  These  results  were  all 
attained  at  a  significance  level  of  x  =  0.05  or  better. 

In  terms  of  recognition  accuracy,  the  results  were 
better  tnan  the  author  expected.  Poorer  results  were 
expected  because  short  phrases  consisting  of  several 
utterances  were  used  rather  than  simple  one  or  two  utterance 
commands.  It  was  anticipated  that  subjects  wculd  run  words 
together  more  than  they  actually  did,  and  it  was  also 
anticipated  that  the  T500  wculd  have  mere  trouble  with 
similar  sounding  terms  such  as  tfIG-25  F0X3AT  and  PIG-Z5R 
FOX BAT . . .or  CHARI I Z  I  CLASS  and  CHARIIZ  II  CIASS.  Though 
the  T602  did  misrecognize  such  words  at  times,  subjects 
cuickly  adapted  to  the  situation,  emphasizing  the  portion  of 
the  utterance  that  was  unique,  thereby  achieving 
better  results.  The  97%  overall  recognition  accuracy 
would  likely  improve  with  practice  and  increased  usage. 
Additionally,  new  high-speed  recognition  systems,  like 
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Threshold's  QUIChTAIK  (Trademark),  require  a  much  shorter 
pause  between  utterances,  thus  permitting  the  operator  to 
speak  faster.  CU1CKTALL  is  advertised  to  reach  speeds  of 
1 82  words-per-minute,  and  99 %  accuracy  for  moderately 
trained  speakers.  Vocabulary  structuring  may  also  be 
performed  which  allows  the  system  to  search  only  a  subset  of 
the  vocabulary,  thus  increasing  the  speed  and  accuracy  of 
recognition.  This  system,  as  advertised,  has  twice  the 
speed  of  the  T60Z  used  in  the  experiment. 

Subjects  tended  to  prefer  voice  before  and  after  the 
experiment  (even  more).  Icr  the  vast  majority  of  subjects, 
this  was  the  first  use  of  voice  continuously  for  an  extended 
period  of  time.  Even  though  it  did  net  meet  some  of  their 
more  lofty  expectations,  they  continued  to  give  voice  the 
edge  in  the  subjective  questionnaire,  and  actually 
strengthed  their  opinions  toward  it  on  several  criteria. 

Thus  this  experiment,  though  outside  an  operational 
setting,  supports  further  research  and  possible  applications 
of  ASR  for  imagery  interpretation  reporting  systems,  and 
perhaps  ether  similar  intelligence  and  tactical  commend  and 
control  data  systems.  The  results  are  certainly  not  new, 
but  add  credence  to  the  related  results  achieved  by  F.ATC , 
NPS,  and  others. 

Use  of  the  ARPANET  facilities  in  this 
experiment  demonstrated,  to  a  limited  degree,  that 
reporting  can  be  performed  without  the  benefit  of  a 
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local  host  computer.  This  may  he  very  beneficial  in  the 
future  if  department  of  defense  organizations  want  to 
remotely  query  cr  update  a  common  data  base. 

E.  RICOKKENDATICN  S 
1 .  Research 

The  time  is  perhaps  ripe  for  the  military  to  perform 
some  research  using  voice  data  entry  as  a  keyboard  assist 
for  one  or  mere  of  the  current  imagery  reportin?  systems, 
such  as  TIPI  IIS,  KARRIS,  CATIS,  PACIR,  AIRES,  and  others. 
Ey  beginning  now  to  look  at  the  use  of  voice  for  these 
systems,  the  intelligence  community  nay  he  able  to  identify 
the  specific  questions  needing  to  he  addressed  to  most  fully 
adapt  voice  as  an  input  modality.  In  the  next  five  cr  ten 
years,  the  outlook  for  "matter-of-fact"  use  of  voice  is 
good.  By  studying  the  problems  associated  with  training, 
user  acceptance,  physical  interfacing,  vocabulary  size, 
vocabulary  data-base  maintenance,  response  times,  and  other 
areas  now,  voice  will  be  more  easily  applied  later. 

Additionally,  voice  input  may  be  applied  to  ether 
tasks  associated  with  the  other  intelligence  disciplines 
using  interactive  computer-controlled  devices.  Command 
center  applications  are  also  receiving  increased  attention 
as  natural  language  query  systems  coupled  with  graphics 
displays  commanded  by  voice  are  now  a  reality  in  terms  cf 
advanced  applications  technology. 
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All  new  imagery  exploitation  systerrs  "being  developed 
or  modified  snculd  fully  consider  the  benefits  of  voice 
recognition  technology.  Considering  the  three  to  eight  years 
it  takes  to  develop  a  new  system,  it  is  highly  likely  that 
by  the  time  it  is  fielded,  significantly  more  voice 
capabilities  will  be  available.  Special  consideration 
should  be  given  to  not  only  to  how  it  might  aid  interpreters 
in  the  reporting  process,  but  also  hew  they  might  be  able  to 
use  it  to  enhance,  manipulate,  annotate,  and  otherwise 
modify  digital  softcopy  imagery  on  systems  such  as  Compass 
Preview . 

2 .  Applications 

Practical  applications  using  voice  data  entry  on  a 
large  scale  will  require  a  significant  amount  of  work.  It 
rust  also  be  proven  that  while  voice  may  be  as  fast  or 
faster  than  typing  that  the  time  differential  achieved 
contributes  commensurately  with  the  additional  cost  of  such 
new  technology.  Careful  attention  must  be  paid  to  involving 
the  users,  since  they  will  ultimately  "sell"  the  system, 
even  though  proven  in  the  lab. 

The  author  recommends  a  small  application  first  with 
a  few  of  the  best  interpreters  who  know  the  imagery 
system  well,  and  are  ambivalent  regarding  vcice  data  entry. 
By  allowing  them  to  use  voice  on  a  daily  basis,  they  can 
develop  the  in-house  expertise  at  the  level  needed  to  apply 
it  or.  a  large  scale  later. ..or  they  may  be  able  to  assess 
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that  it  just  won't  work  for  that  particular  application. 

The  military  is  fortunate,  having  excellent  research 
people  involved  with  voice  technology.  RABC  and  NPS  are 
just  two  military  institutions  able  to  provide  consultation 
and  assistance. 

C.  CONCLUSIONS 

Since  1972,  automatic  speech  recognition  has 
proven  to  be  valuable  for  a  number  of  limited 
applications.  The  future  for  the  technology  is  bright. 
The  author  concludes  voice  is  not  only  feasible,  but 
desirable  as  a  means  toward  the  best  imagery 
interpretation  reporting  possible.  It  is  not  so  much  a 
question  of  whether  voice  can  be  used,  but  rather  ... 
how  can  it  be  used?... how  extensively  can  it  be  used?... and 
how  cost-effective  will  it  be? 
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APPENDIX  A 

USSR/WARSAW  PACT  ORDER  OP  BATTLE  (.OB )  VOCAEULARY 

INSTRUCTIONS:  TRAIN  THE  WORDS  IN  THE  GIVEN  SEQUENCE,  USING 

THE  GIVEN  PROMPT.  WORD  NUMBERS  MARKED  WITH  AN  ASTERISK  MAY 
BE  TRAINED  WITH  THE  GIVEN  PROMPT  OR  YOU  MAY  USE  YOUR  OWN. 
(THESE  WORDS  WILL  BE  USED  FOR  TEXT  EDITING,  AND  THUS  SHOULD 
BE  FAMILIAR ,  EASY  TO  REMEMBER)  ****  BE  SURE  TO  WRITE  IN  THE 
ONE  THAT  YOU  USE  ON  TEE  VOCABULARY  LISTING  SO  THAT  YOU  MAY 
EAVE  IT  FOR  FUTURE  REFERENCE.  **** 


WORD 

PROMPT 

OUTPUT 

e 

ZERO 

0 

1 

ONE 

1 

2 

TWO 

c. 

3 

THREE 

• z 

w 

4 

FOUR 

4 

c 

FIVE 

K 

6 

SIX 

6 

7 

SEVEN 

7 

S 

EIGHT 

8 

9 

NINE 

9 

10 

ALPHA 

A 

11 

BRAVO 

B 

12 

CHARLIE 

C 

13 

DELTA 

D 

14 

ECHO 

E 

15 

FOXTROT 

j 

16 

GOLF 

G 

17 

HOTEL 

H 

18 

INDIA 

I 

19 

JULIET 

J 

20 

KILO 

K 

21 

LIMA 

L 

r  r> 

£.£. 

MIKE 

M 

23 

NOVEMBER 

N 

24 

OSCAR 

0 

25 

POPPA 

P 

26 

QUEBEC 

0 

27 

ROMEO 

R 

28 

SIERRA 

S 

29 

TANGO 

T 

30 

UNIFORM 

U 

31 

VICTOR 

V 

32 

WHISKEY 

w 

33 

XRAY 

X 

96 


34 

YANKEE 

V 

* 

3b 

ZULU 

Z 

36 

POSSIBLE 

POSSIBLE 

3? 

PROBABLE 

'prgbable" 

36 

CONFIRMED 

.CONFIRMED, 

39 

DASH 

- 

42* 

ERASE 

EKSP  <CTRL  A> 

41 

C-0  OR  CARRIAGE  RETURN 

^CARRIAGE  RETUR 

42 

SLASH 

/ 

43* 

KILL  WORD 

<CTRL  W> 

44* 

KILL  LINE 

<,C  TRL  X> 

45* 

REPEAT  LINE 

<CTRL  R> 

46 

SPACE 

<S PACE  CHARACTE 

47 

TEN 

12 

46 

INSTALLATION 

INSTALLATION 

49 

ELEVEN 

11 

50 

UPPER  LEFT 

UPPER  LEFT 

£1 

TANKS 

TANKS 

52 

LIGHT 

LIGHT 

e,t 

w  W 

MEDIUM 

MEDIUM 

£4 

HEAVY 

HEAVY 

55 

T72 

T-72 

56 

Tc2 

T-62 

£7 

T54/55 

T-54?££ 

5e 

T1Z 

T-1Z 

59 

T34/S5 

T-34?85 

60 

TWELVE 

12 

61 

PT76 

PT-76 

62 

AMPEIBEOUS 

AMPHIBEOUS 

63 

UPPER  RIGHT 

UPPER  RIGHT 

64 

APC 

APC 

65 

AT  GW 

AT  GW 

66 

ERDM 

BRDM 

67 

BTR60PK 

BTR-60PK 

ee 

BKP76PB 

BMP-76PB 

69 

BTR152 

BTR-152 

70 

BTR50PK 

BTR-52PK 

71 

FIELD  EWTZRS 

FIELD  EOWITZERS 

72 

ASU65  i 

ASU-65 

73 

SU100 

SU-120” 

74 

AIRBORNE 

AIRBORNE 

75 

LOWER  LEFT 

LOWER  LEFT 

76 

D32 

D-30 

77 

AT3  SAGGER 

AT-3-SAGGER 

78 

ANTI-TK  GUNS 

ANTI-TANK  GUNS 

79 

D74 

D-74 

£0 

D20 

D-22" 

81 

M1955 

M-1955 

82 

D44 

D-44 

63 

BM21 

EM-21 

64 

M1976 

M-1976 

97 


r-~ 

■ 

£5 

BM24 

3M-24 

- 

t  c6 

'  S7 

FR0G3 

frog-3 

FRCG4 

*  R 

I  bB 

FROG? 

FROG-7 

:  £9 

SCUD  A 

SCUD-A 

90 

SCUD  B 

SCUD-B 

91 

SS 12  SCL3RD 

SS -12  SC ALZ30ARD 

92 

SSM 

SSM 

H  53 

ATI  SNAPPER 

AT-1  SNAPPER 

i  94 

65  MILIMETZR 

65— MM 

95 

100  MILIMETR 

100-MM 

96 

SA4  GANEP 

SA-4  GAME 

i  97 

SA6  GAINFUL 

SA-6  GAINFUL 

V  33 

SA6  GECKO 

SA-6  GECKO 

►J  99 

SAS  GASKIN 

SA-9  GASKIN 

100 

LAUNCHERS 

LAUNCHERS 

:f  101 

THIRTEEN 

13 

H  122 

ASV 

ASV 

rj  122 

104 

FOURTEEN 

14 

AA  GUNS 

AA  GUNS 

L>  105 

FIELD  GUNS 

FIELD  GUNS 

t  106 

ZU23/2 

ZU-23/2 

127 

ZSU23/4 

ZSU-23/4 

3  106 

ZSU57/2 

ZSU-57/2 

^  109 

k  lie 

see 

s-6e 

M44 

M-44 

y  in 

M49 

M-49 

4  112 

57  MILIMETER 

57-MM 

]  113 

SU15  FLAGON 

SU-15  FLAGON 

114 

YAK28P  FRBAR 

YAK-28P  FIREBAR 

115 

TU28P  FIDLR 

TU-28P  FIDDLER 

116 

MIG19  FARMER 

MIG-19  FARMER 

h  u7 

MIG21  FSHBED 

MIG-21  FISHBFD 

a  ns 

MIG23  FLGGER 

MIG-23  FLOGGER 

4  H9 

MIG25  FOXBAT 

MIG-25  FOXBAT 

,  122 

MIG27  FLGGER 

MIG-27  FLOGGER 

y  121 

TU20  BEAR 

TU-20  BEAR 

122 

TU126  MOSS 

TU-126  MOSS 

123 

SU9  FISHPOT 

SU-9  FISHPOT 

124 

MIG25R  FXBAT 

MIG-25R  FOXBAT 

125 

TU22  BLINDER 

TU-22  BLINDER 

126 

TU16  BADGER 

TU-16  BADGER 

127 

TU26  BACKFIR 

TU-26  BACKFIRE 

126 

MI4  EOUND 

MI-4  EOUND 

129 

MI12  HOMER 

MI -12  HOMER 

130 

MI6  HOOK 

MI -6  HOOK 

131 

MI8  HIP 

MI-6  HIP 

132 

MI10  HARKE 

MI-10  HARKE 

j  133 

MI24  HIND 

MI-24  HIND 

134 

IL36  MAY 

IL-38  MAY 

135 

M-4  BISON 

M-4  EISONl 

L .... 

96 

j 

136 

SU19  FENCER 

SU-19  FENCER 

137 

FIFTEEN 

15 

136 

ANS  CAP? 

aN-S  CAMP 

139 

AN  12  CUB 

AN-12  CUB 

142 

AN22  COCK 

AN-22  COCK 

141 

AN26  CURL 

AN-26  CURL 

142 

KA15  HjtN 

KA-15  HEN 

143 

KA16  HOG 

XA-1S  HOG 

144 

KA25  HORMONE 

KA-25  HORMONE 

145 

IL12  COACH 

IL-12  COACH 

146 

IL14  CRATE 

IL-14  CRATE 

147 

IL28  BEAGLE 

IL-28  BEAGLE 

146 

IL76  CANDID 

IL-76  CANDID 

149 

AWACS 

AWACS 

150 

3E12  MAIL 

BE-12  MAIL 

151 

TRANSPORTS 

TRANSPORTS 

152 

FIGHTERS 

FIGHTERS 

153 

BOMBERS 

BOMBERS 

154 

FIGKTER-BhRS 

FIGHTER-BOMBERS 

155 

STRIKE/ATTCK 

STRIKE/ATTACK 

156 

HELICOPTERS 

HELICOPTERS 

157 

RECONNAISNC 

RECONNAISSANCE 

156 

SS 

SS 

159 

FRIGATE 

FRIGATE 

160 

SSB 

SSB 

161 

SSGN 

SSGN 

162 

SSBN 

SSBN 

163 

CARRIER 

CARRIER 

164 

CRUISERS 

CRUISERS 

165 

DESTROYERS 

DESTROYERS 

166 

MINESWEEPERS 

MINESWEEPERS 

167 

FRIGATES 

FRIGATES 

168 

CORVETTES 

CORVETTES 

169 

MISSLE 

MISSLE 

170 

TORPEDO 

TORPEDO 

171 

BOATS 

BOATS 

172 

LANDING 

LANDING 

173 

SIXTEEN 

16 

174 

INTELLIGENCE 

INTELLIGENCE 

175 

SHIPS 

SHIPS 

176 

SEVENTEEN 

17 

177 

EIGHTEEN 

16 

178 

KIEV  CLASS 

KIEV  CLASS 

179 

MOSKVA  CLASS 

MOSKVA  CLASS 

180 

SSN 

SSN 

161 

DELTA  CLASS 

DELTA  CLASS 

162 

DELTA2  CLASS 

DELTA  II  CLASS 

163 

H0TEL2  CLASS 

HOTEL  II  CLASS 

184 

H0T2L3  CLASS 

HOTEL  III  CLASS 

185 

ASU57 

ASU-57 

186 

VICTOR  CLASS 

VICTOR"CLASS 

99 


187 

YANKEE  CLASS, 

YANKEE  CLASS 

ise 

G0LF1  CLASS 

GOLF  I  CLASS- 

189 

G0LF2  CLASS 

GOLF  II  CLASS 

19? 

ZJLU4  CLASS 

ZULU  IV  CLASS 

191 

KRSSTA1  CLAS 

KRESTA  I  CLASS 

192 

XHESTA2  CLaS 

KRESTA  II  CLASS 

193 

MIRKA1  CLASS 

MIRKA  I  CLASS 

194 

MIRKA2  CLASS 

MIRKA  II  CLASS 

195 

PETYA1  CLASS 

PETYA  I  CLASS 

196 

PETYA2  CLASS 

PETYA  II  CLASS 

197 

JULIET  CLASS 

JULIET  CLASS 

196 

LOWER  RIGHT 

LOWER  RIGHT 

199 

122  MILIMETR 

12 2 -MM 

200 

FOXTROT  CLAS 

FOXTROT  CLASS 

201 

ROMEO  CLASS 

ROMEO  CLASS 

202 

SSG 

SSG 

203 

BRAVO  CLASS 

ERAVO  CLASS 

204 

ECEOl  CLASS 

ECHO  I  CLASS 

205 

ECH02  CLASS 

ECHO  II  CLASS 

206 

152  MILIMETR 

152-MM 

207 

TANGO  CLASS 

TANGO  CLASS 

208 

WHISKEY  CLAS 

WHISKEY  CLASS 

209 

CEARLIE1  CLS 

CHARLIE  I  CLASS 

210 

CHARLIE2  CLS 

CHARLIE  II  CLASS 

211 

KARA  CLASS 

KARA  CLASS 

212 

SVERDLOV  CLS 

SVERDLOV  CLASS 

213 

KYNDA  CLASS 

KYNDA  CLASS 

214 

KRIVAK  CLASS 

KRIVAK  CLASS 

215 

KASEIN  CLASS 

KASHIN  CLASS 

216 

240  MILIMETR 

240-MM 

217 

KANIN  CLASS 

KANIN  CLASS 

218 

INTERCEPTORS 

INTERCEPTORS 

219 

KOTL1N  CLASS 

KOTLIN  CLASS 

220 

KOTLN  SAM  CL 

KOTLIN-SAM  CLASS 

221 

SKORY  CLASS 

SKORY  CLASS 

222 

RIGA  CLASS 

RIGA  CLASS 

223 

GRISHA  CLASS 

GRISHA  CLASS 

224 

NANUCHKA  CLS 

NANUCHKA  CLASS 

225 

POTI  CLASS 

POTI  CLASS 

226 

OSA1  CLASS 

OSA  I  CLASS 

227 

0SA2  CLASS 

OSA  II  CLASS 

228 

KOMAR  CLASS 

KOMAR  CLASS 

229 

STENKA  CLASS 

STENKA  CLASS 

230 

NINETEEN 

19 

231 

TWENTY 

22 

232 

S HERS  HEN  CLS 

SHERSHEN  CLASS 

233 

TWENTY-ONE 

21 

234 

NATYA  CLASS 

NATYA  CLASS 

235 

YURXA  CLASS 

YURKA  CLASS 

236 

ALLIGATOR  CL 

ALLIGATOR  CLASS 

237 

POLNOCNY  CLS 

POLNOCNY  CLASS 

1 0e 


236 

TWENTY  -TM) 

22 

239 

PRIMORYE  CLS 

PRIMORYE  CLASS 

242 

TWENTY-THREE 

23 

241 

TWENTY-JOUR 

24 

242 

SS16 

SS-16 

243 

SS20 

SS-20 

244 

SS14  SCPGOAT 

SS -14  SCAPEGOAT 

245 

SS15  SCROOGE 

SS -15  SCROOGE 

24c 

ICBM 

ICBM 

247 

IRBM 

IRBM 

248 

MOBILE 

MOBILE 

24:9 

M242 

M-240 

250 

MORTARS 

MORTARS 

251 

ASSAULT  GUNS 

ASSAULT  GUNS 

252 

ROCKET  LCHRS 

ROCKET  LAUNCHERS 

253 

AIRCRAFT 

AIRCRAET 

254 

TWENTY-JIVE 

25 
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APPENDIX  3 


SCENARIO  CARDS 

TYPING  CARDS  ->>>>>>>>  FIRST  TWELVE 
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INSTALLATION  2128-T13213  ** 

!  / 

2  CONFIRMED  KRESTA  II  CLASS  CRUISERS 
3  CONFIRMED  ARESTA  I  CLASS  CRUISERS  ** 

i 

**  l  POSSIBLE 'TANGO  CLASS  SS 

i 

I 

1 

I 

I 

~*S“l2_CGNFfRM£F¥ETSKEf~CLASS"SS 

I 

2  PROBABLE  CEARLIE  II  CLASS  SSGN  ** 

I 

» 

I 

1  CONFIRMED  CHARLIE  I  CLASS  SSGN 

**  { 

i 

i 

^ ,  i _ , _ _ _ 

—  i 


INSTALLATION  0298-114218 

j  ** 

50  CONFIRMED  ASU-65  AIREORNE  ASSAULT  GUNS 

27  CONFIRMED  ASU-57 ' AIRBORNE  ASSAULT  GUNS 

*#  j 

i 

i 

i 

j 

i 

_ _ _ i _ _ _ _ _ 

* 

**  20  POSSIBLE  M— 240  HEAVY  MORTARS 

j 

62  PROBABLE  122-MM  B-30  FIELD  HOWITZERS  ** 

i 

i 

48  CONFIRMED  240-MM  BM-24  ROCKET  LAUNCHERS 

i 

I 


1 

I 
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INSTALLATION  0627-T21253 
6  CONFIRMED  FOXTROT  CLASS  SS  ** 

{  ** 

12  CONFIRMED  JULIET  CLASS  SSG 
**  2  PROBABLE  DELTA  II  CLASS  SSBN 

i 

3  PROBABLE  DELTA  CLASS  SSBN 

**  j 

4~CON ?TRmId"GOLF ' IT”  LAS  S~SS  BN_**  ' 


##  J 


5  CONFIRMED  POTI  CLASS  CORVETTES 


'**  2  POSSIBLE  YANKEE  CLASS  SSBN 


7  PROBABLE  ROMEO  CLASS  SS  ** 


INSTALLATION  0405-T22217 

!  ** 


40  CONFIRMED  T-10  HEAVY  TANKS 


**  57  CONFIRMED  T-34/85  MEDIUM  TANKS 

**  43  CONFIRMED  T -54/55  MEDIUM  TANKS 

i 
I 
« 

I 

!  3_CQN FTRMED~PT-?6~LTGET_AMPETBE0US_TANK S  ' 


**  8  CONFIRMED  BTR-152  APC 


6  CONFIRMED  BRDM  RECONNAISSANCE  APC 


104 

\ 

3 

I 


^ggjggi 


INSTALLATION  0352-T23224  ** 

!  / 

11  CONFIRMED  TU-22  BLINDER  BOMBERS 

i 

l 

I 

20  CONFIRMED  TU-26  BACKFIRE  BOMBERS  ** 

5  PROBABLE  IL-28  BEAGLE  BOMBERS 

;  #* 

i 

_ _ i _ _ _ 

j 

**  2  CONFIRMED  IL-76  CANDID  TRANSPORTS 

i 

i 

j 

15  CONFIRMED  AN-12  COB  TRANSPORTS 

i 

i 

} 

7  CONFIRMED  MI-8  HIP  HELICOPTERS 

i 

i 

_ _ _ i _ _ _ 


INSTALLATION  0247-T24283 


**  5  PROBABLE  KOMAR  CLASS  MISSLE  BOATS 


17  CONFIRMEE  OSA  I  CLASS  MISSLE  BOATS 


5  CONFIRMED  OSA  II  CLASS  MISSLE  BOATS 


**  7  CONFIRMED  STENKA  CLASS  TORPEDO  BOATS 


** 


11  POSSIBLE  NANUCHKA  CLASS  TORPEDO  BOATS 


6  POSSIBLE  GRISHA  CLASS  CORVETTES  ** 

2  PROBABLE  SHEBSHEN  CLASS  TORPEDO  EOATS**! 


105 


INSTALLATION  2243-T31278 

##  j 

12  CON  FIRMED  MIS-27  FLCGGIR  STRIKE/ATTACK 
AIRCRAFT  1 

16  CONFIRMED  SU-19  FENCER  STRIKE/ATTACK 
AIRCRAFT  i  ** 

i 

2  POSSIBLE  MIG-25R  FGXBAT  RECONNAISSANCE 
AIRCRAFT  **  i 


**  1  CONFIRMED  IL-36  MAY  ASW  AIRCRAFT 

**  j 

3  CONFIRMED  AN-8  CAMP  TRANSPORTS 

i 

i 

£  CONFIRMED  AN-26  CURL  TRANSPORTS 

j  ** 

i 

i 


i  — —  —  i 


INSTALLATION  2657-T32179  ** 

!  /  ** 

2  CONFIRMED  HOTEL  II  CLASS  SSBN  / 

1  CONFIRMED  HOTEL  III  CLASS  SSBN 

i 

i 

i 

t 

i 

i 

i 

i 

i 

_ i _ 

**  j 

1  PROBABLE  GOLF  I  CLASS  SS3 

*#  j 

1  PROBABLE  MIMA  I  CLASS  LIGHT  FRIGATE 

i 

i 

|  ** 

1  POSSIBLE  ZULU  IV  CLASS  SS 

i 

i 

i 

i 

_ _ _ _ _ _  i _ _ _ _ _ 

i 


INSTALLATION  0410-T33252 

**  4  CONFIRMED  100-MM  M-49  AA  GUNS 

4  CONFINED  ZSU-57/2  AA  GUNS - ** 

6  CONFIRMED  65-MM  M-44  AA  GUNS 

i  ## 

; 

/  ! 

8  CONFIRMED  FROG-4  SSM  MOBILE  LAUNCHERS 


i 

i 

i 

6  PROBABLE  AT-1  SNAPPER  ATGW 

! 

| 

4  CONFIRMED  122-MM  D-74  FIELD  GUNS 

i 

j 

21  CONFIRMED  85-MM  D-44  ANTI-TANK  GUNS 

__  ___  _ _ _ _ _ _ _ _ _ 

-  —  —  |  _  —  — 


INSTALLATION  0173-T34246  **  ** 

1  CONFIRMED  TU-126  MOSS  AWACS  / 

1  CONFIRMED  TU-16  BADGER  RECONNAISSANCE 
AIRCRAFT  1 

16  CONFIRMED  AN-22  COCK  TRANSPORTS 

**  I 


i 

j 

"li'cQN FIRM ED~TU -20 ~EEAR 'BOMBERS* 


/ 


12  CONFIRMED  TU-22  BLINDER  BOMBERS  ** 


2  CONFIRMED  TU-20  BEAR  RECONNAISSANCE 


AIRCRAFT 
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VC  ICE-  UNBUFFERED  CARDS  ->  >  >  NEXT  TWELVE 


!  INSTALLATION  0156-VUL50  ** 

I  I  / 

19  PROBABLE  YAK-28P  FIREBAR  FIGHTER-BOMBERS 

I  *"  I 

I**  12  CONFIRMED  SU-15  FLAGON  INTERCEPTORS 

I  I  “ 

! 20  CONFIRMED  TU-28P  FIDDLER  INTERCEPTORS 

i  !  ** 

I  !  ** 

i~T3"PR6RnLOlG-25lF0XBAT”lNTERCZ?TCRS~"“ 

i  i 

'  i 

i  i 

!  I 

!  11  POSSIBLE  SU-9  FISRPOT  FIGHTERS  ** 


15  PROBABLE  MIG-21  FISHBID  FIGHTERS** 


INSTALLATION  2357-712252 

i 

1  CONFIRMED  MOSXVA.CLASS  CARRIER 

! 

**  ; 

1  CONFIRMED  KIEV  CLASS  CARRIER 

“  i 

**  2  PROEABLE  KARA_CLASS  CRUISERS 

__________ i _____________________ 

j 

**  3  CONFIRMED  KASHIN_CLASS  DESTROYERS 

i 

**  4  CONFIRMED  KRIVAK  CLASS  FRIGATES 

i  ~ 
i 

; 

8  CONFIRMEE  MIRKA  II  CLASS  LIGHT  FRIGATES 

“  i  ~ 
i 

_ _ _ _ _ i _ _ _ _ 
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INSTALLATION  01S8-V13259  ** 

6  PROBABLE  57-MM  S-60  MEDIUM  AA  GUNS 

j  - 

*#  4  CONFIRMED  SA-8_GECK0  LAUNCHERS 

i 

t 

i 

3  CONFIRMED  SA-4.GANEF  LAUNCHERS  ** 

i 

i 

_ _ i _ _ _ 

_  **  j 

4  CONFIRMED  SA-6_GAINFUL  LAUNCHERS 

3  CONFIRMED  SS-12  SCALEBCARD  MOBILE  SSM 

!  * 

5  CONFIRMED  FROG-3  MOBILE  SSM  ** 

■ 

i 

i 

**  4  CONFIRMED  SA-9' GASKIN  LAUNCHERS 


INSTALLATION  0199-V14197 

[  ** 

16  CONFIRMED  MI-4  HOUND  HELICOPTERS 

i 

i 

**  j 

11  CONFIRMED  MI-12_EOMER  EELICOPTERS 

i 

**  5  PROBABLE  MI-6_H00K  HELICOPTERS 

_ _  _ _ _  _  i _ _ _ _ 

i 

i 

;  ## 

21  CONFIRMED  MI-10_EARKE  HELICOPTERS 

i 

i 

i 

##  | 

19  PROBABLE  MI-24_KIND  HELICOPTERS 

i 
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INSTALLATION  0208-V21221 

{ ## 

1  CONFIRMS!)  SS-16  MOBILE  ICBM 

| 

**  1  POSSIBLE  SS-1&  SCROOGE  MOBILE  IRBM 
**  1  CONFIRMED  SS-14  SCAPEGOAT  MOBILE  IRBM 
2  PROBABLE  SS-20  MOBILE  IRBM  ** 


**  1  CONFIRMED  FROG-?  SSM 

**  3  CONFIRMED  SCUD  A  SSM 

!  - 

1  POSSIBLE  SCUD_B  SSM 

I 

I 

I 

I 


INSTALLATION  0195-V22231 


10  CONFIRMED  KA-25  HORMONE  HELICOPTERS 


!  ** 


11  CONFIRMED  MI-6  HI?  HELICOPTERS 

T 


**  4  CONFIRMED  KA-15  HEN  HELICOPTERS 


**  6  CONFIRMED  KA-16  HOG  HELICOPTERS 


22  CONFIRMS!)  IL-12  COACE  TRANSPORTS 


22  CONFIRMED  IL-14  CRATE  TRANSPORTS  ** 


_ _ I  _ _ _ _ _ i 


I 


I 


INSTALLATION  3327-V23249 
**  2  PROBABLE  PRIMORYE  CLASS  INTELLIGENCE 
SEIPS  ! 

I  $$ 

3  CONFIRMED  POLNOCNY  CLASS  LANDING  SHIPS 
2  CONFIRMED  ALLIGATOR  CLASS  LANDING  SHIPS 

|  ** 

f 

i 

i 


3  CONFIRMED  YUEKA  CLASS  MINESWEEPERS 


i 

» 

i 

i 


2  POSSIELE  NATYA  CLASS  MINESWEEPERS 


i 

i 


4  PROBABLE  PETYA  I  CLASS  FRIGATES 

~  T 

i 

( 

i 


INSTALLATION  0167-V2427? 

**  60  CONFIRMED  BTR-60PK  AMPHIBEOUS  APC 
**  25  CONFIRMED  T-62  MEDIUM  TANKS 

i 

23  CONFIRMED  85-Mh  D-44  ANTI-TANK  GUNS  ** 

i 

i 

j  ** 

18  PROBABLE  BM-21  ROCKET  LAUNCHERS 

i  ~ 

_ _ i  __ _ _ _ _ _ 

i 

i 

i 

**22  CONFIRMED  122-MM  D-3e  FIELD_HOWI TZERS 

i 

\ 

**  ( 

19  CONFIRMED  M-1955  FIE1D_H0WITZERS 

17  CONFIRMED  M-1976  AIRBORNE  ** 
ASSAULT_GUNS  { 

____________________  »  __________ _ _______ _ 

i 
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i  INSTALLATION  0528-V31176 

I  j 

!  **  11  CONFIRMED  PETYA  II_CLAS S  FRIGATES 

(  i  “ 

!  **  2  PROBABLE  BRAVO  CLASS  SS 

i  i 


t 

3  CONFIRMS!  ECEO.I.CLASS  SSGN 

_  _____  _  _  i _ _ _ _ __ 

— —  —  j 

i 

|  ## 

12  CONFIRMED  ECHO  II  CLASS  SSGN 

“  i 

i 

j  ** 

5  CONFIRMED  RIGA  CLASS  FRIGATES 

i 

i 

i 

i 

i 

i 

_  ____ _ i  . _ _ 

i 


i  _ _ _ t _ _ _ _ _ * 

INSTALLATION  0410-V32237  ** 

22  CONFIRMED  BTR-50PK  AMPEIBEOUS  APC 


40  CONFIRMED  T-72  HEAVY  TANKS  —  ** 
18  PROBABLE  SU-100  ASSAULT  GUNS 
#*  ! 


i**2£  CONFIRMED  152-MM  D-20  FIELD_HOWI TZERS 

i _ i _ _ _ l 

*#  { 

24  CONFIRMED  100-MM  M-1955  FIELD_GUNS 


13  PROBABLE  M-1976  AIRBORNE  ASSAULT  GUNS 
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INSTALLATION  0276-V33264 
15  CONFIRMEE  MIG-21  FISHBEE  FIGHTERS 

| 

j 

12  CONFIRMED  MIG-19  FARMER  FIGHTER-BOMBERS 

”  i 
i 

t 

**  11  PROBABLE  MIG-23  FLOGGER  FIGHTERS 

i — 

_ i  _ 

j 

17  CONFIRMEE  MIG-27  FLOGGER  STRIKE /ATTACK 
AIRCRAFT  T 

i 

i 

*#  | 

21  CONFIRMEE  MIG-25  FOXBAT  INTERCEPTORS 

T 

I 

**  i 

3  POSSIBLE  TU-28P  FIEELER  INTERCEPTORS 


INSTALLATION  0362-V34273 

i 

**  2  PROBABLE  SKORY ' CLASS  DESTROYERS 

T 

**  3  CONFIRMED  KOTLIN.CLASS  DESTROYERS 

i 

2  CONFIRMED  KYNDA.CLASS  CRUISERS  ** 

5  CONFIRMED  KANIN  CLASS  DESTROYERS 

—  { 

2  CONFIRMED  SVERDLOV_CLASS  DESTROYERS 
**  } 


2  PROBABLE  SHSRSHEN  CLASS  TORPEDO  BOATS 


4  CONFIRMED  KOTLIN  SAM  CLASS  DESTROYERS 

T  *# 

_ _ _ i  _ _ _ i 


I 
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INSTALLATION  0613-V51214 

j 

**  4  CONFINED  EMP-76PE  APC 

7  CONFIRMED  ER DM  APC  ** 

I 

3  CONFIRMED  AT-3  SAGGER  ATGW  ** 

~  i 

**  4  PROBABLE  ZSU-23/4  AA_GUNS 

t  . 

__  _  _  l  _  _  ' _ _ _ _ _ _ _ 

.  w  —  -  —  , 

42  CONFIRMED  T-54/53  MEDIUM  TANKS  ** 

I 

j  *# 

4  PROBABLE  SA-9.GASKIN  LAUNCHERS 

I 

I 

f 

j  ** 

6  PROBABLE  ZU-23/2  AA  GUNS 

_____  i  ___ 

i 


INSTALLATION  0115-V52314 

**  t  CONFIRMED  M-4  BISON  BOMBERS 

“  i 

**  1  POSSIBLE  TU-2B  BEAR  RECONNAISSANCE 
AIRCRAFT  T 

j 

12  CONFIRMED  TU-20  BEAR  BOMBERS  ** 

**  1  CONFIRMED  TU-126  MOSS  AW  ACS 

i 

_  _  i _ 

j 

2  CONFIRMED  BE-12  MAIL  RECONNAISSANCE 
AIRCRAFT  i 

i 

7  CONFIRMED  II-2E  BEAGLE  BOMBERS  ** 

17  CONFIRMED  TU-16_BADGER  BOMBERS  ** 

i 

**  3  PROBABLE  TU-16 ' BADGER  RECONNAISSANCE 
AIRCRAFT 
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j _ _ _ _ _ _ _ _ 

INSTALLATION  0126-V53213  ** 

{  / 

2  CONFIRMED  XRESTA  II  CLASS  CRUISERS 
3  CONFIRMED  OESTA  I  CLASS  CRUISERS  ** 

I  ~ 

**  1  POSSIBLE1 TANGO_CLASS  SS 

! 

i 

i 

i 

^*~12_C ON  FIRMED ~WHfS ASYlCLASS“SS 

i 

2  PROBABLE  CHARLIE  II  CLASS  SSGN  ** 

i  ~ 

i 

i 

i 

i 

1  CONFIRMED  CHARLIE  I_CLASS  SSGN 

j 

t 

i 

_ _ _ _ _ __ _ i  _  __ _ ________ _ ____ 

i 


_ _ _ _ i _ _ _ _ 

INSTALLATION  0296-V54216 

|  ** 

£0  CONFIRMED  ASU-85  AIRBORNE  ASSAULTJJUNS 

i 

27  CONFIRMED  ASU-57 ‘ AIRBORNE  ASSAULT_GUNS 

**  | 

i 

i 

i 

i 

i 

_ _ _ _ _ i _ _ _ _ 

~  i 

**  20  POSSIBLE  M-240  HEAVY  MORTARS 

i 

62  PROBABLE  122-MM  D-30  FIELD  HOWITZERS  ** 

i 

i 

i 

48  CONFIRMED  240-MM  BM-24  ROCKET-LAUNCHERS 

! 

**  j 

_______________ i 

t 


lie 


9 


INSTALLATION  0827-V61253 
6  CONFIRMEE  FOXTROT_CLASS  SS  ** 

j  s. 

12  CONFIRMED  JULIET  CLASS  SSG 
**  2  PROBABLE  DZLTA_I I_CLASS  SSEN 

j 

3  PROBABLE  DELTA_CLASS  SSEN 

**  j 

4~c6nfTrmed_g6lf  IIIclass~ssbn~55 

I 

**  j 

5  CONFIRMED  PGTI_CLASS  CORVETTES 

i 

**  2  POSSIBLE  YANKEE'CLASS  SSBN 

i 

7  PROBABLE  ROMEO  CLASS  SS  ** 

_ __ _ _ _ i  _ _ _ 


_ i 

INSTALLATION  0405-V62217 

j 

40  CONFIRMED  T-10  HEAVY  TANKS 

j 

**  57  CONFIRMED  T-34/65  MEDIUM  TANKS 

**  43  CONFIRMED  T-54/55  MEDIUM  TANKS 

I 

I 

J 

I 

1 

_ I _ _ _ _ 

3  CONFIRMED  PT-76  LIGHT  AMPHIBECUS  TANKS 

«*  j 

**  6  CONFIRMED  BTR-152  APC 

t 

i 

J  ** 

6  CONFIRMED  BRDtf  RECONNAISSANCE  APC 

i 


i 

i 

i 

i 
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INSTALLATION  0352-V63224  ** 

i  / 

11  CONFIRMED  TU-22  BLINDER  BOMBERS 

i  - 
i 

j 

20  CONFIRMED  TU-26  BACKFIRE  BOMBERS  ** 
5  PROBABLE  IL-28  BEAGLE  BOMBERS-** 


**  2  CONFIRMED  IL-76  CANDID  TRANSPORTS 


| 

15  CONFIRMED  AN-12_CUB  TRANSPORTS 

i 

j  ** 

7  CONFIRMED  MI-S  HIP  HELICOPTERS 

“  i 
i 

_ _ _ _ _ _ i  _ 

i 


_ i 

INSTALLATION  0247-V642S3 

i 

**  5  PRCBAELS  KOMAr'cLASS  MISSLE  BOATS 

i 

i 

** 

17  CONFIRMED  GSA_I_CLASS  MISSLE  BOATS 

i 

5  CONFIRMED  OSA_II  CLASS  MISSLE  BOATS 

**  T 


i 

i 

j 

**  7  CONFIRMED  STENKA  CLASS  TORPEDO  BOATS 

i  - 

i 

#*  ; 

11  POSSIBLE  NANUCHKA.CLASS  TORPEDO  BOATS 

i 

6  POSSIBLE  GRISHa'cLASS  CORVETTES  ** 

2  PROBABLE  SHSRSHEN_CLASS  TORPEDO  BOATS** 

j 
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INSTALLATION  2243-V71278 
**  } 

12  CONFIRMED  MIG-27_FL0GGER  STRIKE/ATTACK 
AIRCRAFT  j 

16  CONFIRMED  SU-19  FENCER  STRIKE/ATTACK 
AIRCRAFT  |  ** 

2  POSSIBLE  MIG-25R  FOXBA1  RECONNAISSANCE 

AIRCRAFT  #*  !  _ _ 


**  1  CONFIRMED  IL-38_MAY  ASW  AIRCRAFT 
>  j 

3  CONFIRMED  AN-8  CAMP  TRANSPORTS 


5  CONFIRMED  AN-26  CURL  TRANSPORTS 

|  ** 


INSTALLATION  0657-V72179  ** 

1  /  * 
2  CONFIRMED  HOTEL  II  CLASS  SSBN  / 
1  CONFIRMED  HOTEL  III  CLASS  SSBN 


1  PROBABLE  GOLF  I  CLASS  SSB 

5*} 

1  PROBABIE  MIRKA  I  CLASS  LIGHT  FRIGATE 


1  POSSIBLE  ZULU  I?  CLASS  SS 


!  INSTALLATION  2410-V73252  ! 

i  **  4  CONFIRMED  10Z-MM  M-49  AA  GUNS  ! 

!  4  CONFIRMED  ZSU-57/2  AA.GUNS  - **  i 

!  5  CONFIRMED  Ss-MM  M-44  AA_GUNS  --*#  ! 

i  i  i 

i  i  i 

;  **  j  J 

!  /  '  ! 

!  S  CONFIRMED  FRCG-4  SSM  MOEILE  LAUNCHERS  ! 

i  i  i 

i _ i _ i 

i  i  i 

j  j  j 

[  6  PROBAELE  AT-1  SNAPPER  ATGW  | 

i  “  i  i 

!  1  I 

j  ##  j  j 

|  4  CONFIRMED  122-MM  D-74  FIELD  GUNS  i 

i  i  i 

!  j  j 

!  21  CONFIRMED  8 5 -MM  D-44  ANTI-TANK  GUNS  ! 

——————————  i  — —  —  —  i 


INSTALLATION  0173-774246  **  ** 

1  CONFIRMED  TU-126_M0SS  AWACS  / 

1  CONFIRMED  TU-16_BADGER  RECONNAISSANCE 
AIRCRAFT  1 

16  CONFIRMED  AN-22  COCK  TRANSPORTS 

j 

i 

i 

i 

i 

j 

'ie“cONFIRMED"‘TU-20_BEAR_BCMBERS 


/ 


12  CONFIRMED  TU-22  BLINDER  BOMBERS  ** 


2  CONFIRMED  TU-20  BEAR  RECONNAISSANCE 


AIRCRAFT 


** 


APPENDIX  C 

1600  TRAINING  INSTRUCTIONS 

For  this  experiment  a  254  word  vocabulary  will  be  used 
with  the  Threshold  600  (1600)  voice  recognition  system.  Tou 
will  be  required  to  speak  each  utterance  ten  times  to  train 
the  T600  to  recognize  your  voice.  Two  sessions  of 
approximately  90  minutes  will  be  required  to  complete  the 
training  prior  to  experimentation. 

Please  observe  the  following  guidelines  during  training 
and  operation  of  the  T600,  as  they  will  improve  performance 
and  reduce  the  time  required  for  retraining. 

a.  Use  variety.  Say  the  repetitions  with  the 
variety  of  intonation,  emphasis,  and  volume 
you  would  expect  to  use  in  normal  speech. 

d.  Speak  crisply  without  pausing.  Se  natural 
and  relaxed.  Don't  exaggerate  or 
overemphasize;  for  example  when  saying  the 
word  "five",  don't  say  "FI-I-VZE” ,  thereby 
overemphasizing  the  end  of  the  word  in  an 
unnatural  way. 
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b.  Do  the  repetitions  in  groups  to  avoid  breath 
noise  and  help  you  count  the  reps.  For 
example  to  train  the  word  "zero"  group  the 
zeros  as  follows: 

000-000-000-0 

or 

000-000-0000 
rather  than  - 

0000000000 

or 

0  —0—0  —0—0  —0  —0  —0—0—0 

c.  Adjust  the  microphone  carefully,  as 
demonstrated  (  see  the  picture). 

e.  Leave  a  distinct  pause  between  words.  You 

must  wait  for  the  green  RIADY  light  to  come 
on  before  saying  the  next  utterance. 

f.  Use  the  proper  volume.  Watch  the  meter; 
the  needle  should  be  in  the  green  area  or 
just  slightly  in  the  red  on  the  peak  parts 
of  the  word.  Words  trained  in  the  lower 
white  or  upper  red  will  give  poorer  results. 
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Cnee  you  are  corf or  table  with  training  the  T62Z,  I  will 
ask  you  tc  operate  the  keyboard  for  the  remainder  of  the 
training.  I  will  renain  nearby  to  provide  assistance  as 
required.  Be  sure  to  ask  for  help  if  you  have  any 
questions.  Take  breaks  as  you  need  them?  a  convenient  place 
to  break  is  every  few  pages. 


Tc  train  a  word  - 


Operating  the  T600 

YOU  TYPE 


CTRL-U 

<word  number > 
.e.g.  0 


####***#*####*#**** 

T6ZZ  RESPONSE 


Cworl  prcmpt> 
ZERO 


Now  you  say  the  word  or  phrase  10  times.  Once  the  current 
phrase  disappears  you  are  ready  to  go  onto  the  next  word  of 
the  vocabulary.  Again  you  type  CTRL-U  and  continue  as 
before . 
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APPENDIX 


1/ 


TYPING  TEST 

THE  SOVIET  NAVAL  AIR  FORCE 

FOR  THE  FIRST  TIKE  IN  ITS  HISTORY,  THE  SOVIET  NAVAL 
AIR  FORCE  WILL  BE  PUTTING  TO  SEA  WITH  ITS  OWN  AIRCRAFT 
EMBARKED  ON  TEE  FIRST  OF  THE  NEW  SOVIET  AIRCRAFT  CARRIERS, 
THE  KIEV,  WHICH  HAL  ALREArY  BEGUN  ITS  WORKING-UP  TRIALS  IN 
THE  AUTUMN  OF  1974.  DISPLACING  SOME  36,020  TONS  WITH  AN 
OVERALL  LENGTH  SLIGHTLY  IN  EXCESS  OF  902  FEET,  THE  KIEV 
IS  PRESUMED  TO  EMBARK  40-60  AIRCRAFT  IN  ALL,  COMPRISING 
A  MIX  OF  HELICOPTERS  AND  FIXED-WING  V/STCL  AIRCRAFT 
(TEE  KIEV  SHOWS  NO  SIGNS  OF  ARRESTER  CABLES  OR  LAUNCH 
CATAPULTS}.  THE  SUGGESTED  VERSION  OF  THE  STRIKE  AND 
RECONNAISSANCE  FIGHTER  TO  BE  EMBARKED  ON  THE  KIEV  IS  TEE 
YAK-36,  A  VERSION  OF  WHICH  WAS  TESTED  ON  THE  AIRFIELDS  NEAR 
MOSCOW  AND  GIVEN  SEA  TRIALS  ON  THE  SOVIET  HELICOPTER- 
CARRIER  MOSKVA.  TEE  YAK-36  UTILIZES  VECTORED  THRUST 
AND  DIRECT  DIET  IN  COMBINATION.  SUCH  AN  AIR  COMPLEMENT 
MIGHT  BE  BROKEN  DOWN  INTO  30  KA-25  ASW  HELICOPTERS  AND 

15-20  V/STCL  FIXED-WING  AIRCRAFT.  HOW  MANY  OF  THESE 
CARRIERS  WILL  BE  PRODUCED  ? 

AT  LEAST  TWO  OF  THESE  KIEV-CLASS  AIRCRAFT  CARRIERS  ARE 
DUS  TO  ENTER  SERVICE,  WITH  THE  POSSIBILITY  OF  THF  SOVIET 
NAVY  PRODUCING  A  WHOLE  CLASS  OF  SOME  6-6  SHIPS,  THEREBY 
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FACILITATING  CONTINUOUS  DEPLOYMENT  OF  ONE  VESSEL  IN  BOTH  TEE 
MEDITERRANEAN  AND  TEE  INDIAN  OCEAN.  TEE  EELICOPTSR 
COMPLEMENT  PROVIDES  INTENSIVE  AS*  CAPABILITY  INTO  DISTANT 
SEA  AREAS  (FOR  DEFENSIVE  AND  OFFENSIVE  PURPOSES),  AS  WELL  AS 
FURNISHING  AIRBORNE  TARGET  GUIDANCE  FOR  SURFACE-TC  SURFACE 
ANTISHIP  MISSLES.  TEE  V/STOL  AIRCRAFT,  WHILE  PROVIDING  A 
STRIKE  CAPABILITY,  MUST  OBVIOUSLY  INCREASE  THE 
RECONNAISSANCE  COVERAGE  OF  THE  SOVIET  NAVAL  AIR  ARM  IN  AREAS 
WHICH  ARE  BEYOND  THE  RANGE  OF  EXISTING  LAND-BASED  AIRCRAFT. 
MEANWHILE,  THE  ARMAMENT  OF  TEE  KIEV-CLASS  SHIPS  IS  ITSELF 
SIGNIFICANT.  IT  CONSISTS  OF  A  TWIN  LAUNCHER  FOR  ASW 
MISSLES,  TWO  12-BARRELL  N:SU  AS  ROCKET  LAUNCHERS,  TWO  SA-N-3 
SAM  TWIN  LAUNCHERS,  A  NUMEER  OF  RETRACTABLE  SA-N-4  SAM 
LAUNCHERS,  MULTIPLE  57-MM  AAA  MOUNTS  AND  SMALLER  WEAPONS  FOR 
CLOSE-IN  PROTECTION  AGAINST  MISSLES  AND  OTHER  GUIDED 
WEAPONS . 
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PP.E/POST  SUBJECTIVE  QUESTIONNAIRE 
Subjective  Ouesticnnaire  Name: 


INSTRUCTIONS:  Express  your  feelings  regarding  typed  data 
entry  and  voice  data  entry.  CIRCLE  TEE  NUMBER  which  BEST 
DESCRIBES  your  opinion  for  each  question. 

1.  Which  data  entry  mcde  do  you  think  is  the  easiest  to  use 
to  enter  character  strings  and  commands? 


Typed 

Da  ta 

Entry 

<= 

<= 

<= 

Neutral 

# 

___  V 
_  y 

=> 

Voice 
Da  ta 
Entry 
=> 

1 

2 

4 

5 

6 

7 

2.  Which  data  entry 

for  entering  character 

mode  do  you 
strings  and 

think  is  the 
commands? 

fas 

test  mode 

Typed 

Data 

Entry 

<= 

<= 

<= 

Neutral 

* 

=> 

=> 

Voice 

Data 

Entry 

=> 

1 

2 

3 

4 

5 

6 

7 

3.  Which  data  entry 

character  strings  and 

mode  is  the 
commends? 

most  accurate 

for 

entering 

Typed 

Data 

Entry 

<= 

<- 

<= 

Neutral 

* 

=> 

=> 

Voice 

Data 

Entry 

=> 

1 

2 

3 

4 

6s 

w 

6 

7 
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4.  Which  lata  entry  node  provides  the  most  flexibility,  in 
general,  for  interaction  with  a  computer? 


Typed 

Data 

Entry 

<=  <  = 

<  = 

Neutral 

* 

=> 

=> 

Voice 

Data 

Entry 

=> 

1  2 

3 

4 

R 

6 

7 

S.  which  data 

several  hours,  if 

entry  mode  would 
required? 

you  prefer 

to  op 

erate  for 

Typed 

Data 

Entry 

<*  <= 

<= 

Neutral 

* 

=  > 

=> 

Voice 

Data 

Entry 

=> 

1  2 

’ X 
w 

4 

5 

6 

7 

6.  'which  data  entry  mode  would  you  prefer 

more  sporadic  user  of  a  computer  system? 

tc  operate  as  a 

Typed 

Data 

Entry 

<=  <= 

Neutral 

* 

=> 

=> 

Voice 

Data 

Entry 

=> 

1  2 

T 

w 

4 

R 

e 

7 

7.  Which  data 

operation? 

entry 

mode  promotes  the 

most 

relaxed 

Typed 

Data 

Entry 

<=  <- 

<  = 

Neutral 

=> 

=> 

Voice 

Data 

Entry 

=> 

1  2 

3 

4 

5 

6 

7 
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S.  Which  data  entry  trode  would  he  the  rrost  advantageous  to 
use  to  update  an  on-line  data  base  cf  Intelligence 
information? 


Typed 

Neutral 

V  oice 

Da  t  a 

Data 

Entry 

Ent  ry 

<= 

<= 

<= 

* 

=> 

=> 

=> 

1 

2 

w 

4 

6 

7 

9.  Which  data 

interface  in 

environment? 

entry  mode  provides  the  best  man- 
a  time-critical,  high-pressure 

-machine 

work 

Typed 

Data 

Entry 

<=  <= 

Neutral 

<=  * 

=>  => 

Voice 

Data 

Entry 

=> 

1  2 

3  4 

5  6 

? 

10.  Which  data 

learn? 

entry  mode  do  you 

think  is  the  easiest  to 

Typed 

Data 

Entry 

O  <* 

Neutral 

<=  * 

=>  => 

Voice 

Data 

Entry 

=> 

1  2 

3  4 

5  6 

7 

12? 
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APPENDIX  P 
SUBJECT  DATA  SEEET 

Subject  Data  Sheet  Date: _ 

Name : _ 

Service: _  Rank/Grade : _ 

Job/Specialty  Description  (last  job  /  next  job) 


Age: _ 


Prior  to  this  experiment  what  has  been  your  experience  with 
voice  data  entry  systems  ?  Check  one  or  more. 

_ a.  I  have  used  a  voice  data  entry  system. 

_ b.  I  have  seen  a  voice  data  entry  system  demonstrated. 

_ c.  I  have  studied  voice  data  entry  systems  (class, 

report,  thesis,  etc.) 

_ d.  I  have  no  experience  with  voice  data  entry  systems. 

If  you  checked  a.  above,  circle  the  term  that  best  describes 
your  experience  and  skill  with  voice  data  entry. 

Experience  -  Skill- 

Considerable  High 

Moderate  Average 

Minimal  Novice 


Explain : 


If  you  checked  c.  above,  please  briefly  state  the  extent 
of  your  studies. 
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APPENDIX  G 


INSTRUCTIONS  BRIEFED  TO  SUBJECTS 
TYPING  MODE 

1.  During  this  portion  of  the  experiment  you  will  view  12 
cards  and  use  the  ADM  terminal  to  write  a  report  on  each 
card  similar  to  the  one  you  saw  in  the  sample  (or  other 
portion  o?  the  experiment).  I  will  step  you  after  every 
four  cards.  This  will  give  you  a  break  and  allow  me  to 
collect  some  data. 

2.  You  will  be  using  a  text  editor  at  the  ISIS  host 
computer.  The  edit  keys  discussed  during  training  which  may 
be  used  are  shown  cn  the  card  at  the  terminal.  You  may  edit 
errors  only  if  you  are  on  the  line  with  the  error  in  it, 
i.e.  if  you  notice  an  error  on  the  previous  line,  do  not 
attempt  to  correct  it.  However,  I  will  demonstrate  hew  you 
may  void  the  previous  line  if  you  wish  to  do  it  over. 

3.  Pencil  and  paper  are  provided  if  you  want  to  use  them  to 
take  notes  as  you  lock  in  the  viewport. 

4.  Now  practice  on  this  card. 

5.  <critique  the  repcrt> 

6.  You  are  to  go  as  fast  as  you  can  while  trying  to 
minimize  errors.  Keep  in  mind  you  are  writing  an 
intelligence  report  which  should  be  timely,  accurate,  and 
complete.  Questions? 

7.  Ck,  start, 
g.  CTrial  #1> 

9.  Ok,  stop.  Rest  a  moment,  then  you  will  do  four  more. 

10.  Ck,  start. 

11.  <Trial  #2> 
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12.  Ok,  step.  Rest  a  moment,  this  is  the  last  set  of  four 
you  will  type  for  the  experiment. 

13.  Ck,  start. 

14.  <Tr ial  »3> 


15.  Stop.  You  deserve  a  break.  Relax  a  while.  You  may 
set  up  and  move  around,  set  a  drink,  etc. 


VC  ICE-UNBUFFERED  HJDE 

1.  During  this  portion  of  the  experiment  you  will  view  12 
cards,  ana  use  the  T600  in  unbuffered  mode  to  write  a  report 
for  each  card  like  the  one  you  saw  in  the  sample  ( cr  other 
part  of  the  experiment).  I  will  stop  you  after  every  four 
cards.  This  will  save  you  a  break  and  allow  me  to  collect 
some  iata. 

2.  The  T620  unbuffered  mode  allows  you  to  send  the  output 
corresponding  to  an  utterance  immediately  to  the  host 
computer.  So  fcr  example,  when  ycu  say  "  CON  FI  HMD  , "  it  is 
sent  immediately  to  the  computer,  and  in  this  case,  becomes 
a  part  of  the  text  in  the  text  editor  at  the  ISIZ  computer. 
Ycu  may  edit  your  input  as  long  as  ycu  are  cn  the  line  that 
has  the  error  using  the  edit  commands  you  trained.  A  list 
of  tne  edit  commands  ycu  use  is  provided  for  you  here,  along 
with  a  list  of  the  vocabulary  as  reference  material. 

2.  If  ycu  look  in  the  viewport  at  this  time,  ycu  will  see 
that  the  three  bottom  lines  of  tne  T€00  display  may  be  seen. 
These  will  provide  a  visual  feedback  of  the  text  editor 
contents,  and  allow  you  to  view  the  editing  process  as  well 
as  the  card. 

4.  Now  practice  using  the  sample  card  provided. 

5.  Ccritique  the  report> 

e.  You  are  to  go  as  fast  as  you  can  while  trying  tc 
minimize  errors.  Keep  in  mind  you  are  writing  an 
intelligence  report  which  should  be  timely,  accurate,  and 
complete.  Questions? 

?.  Ck,  start. 

6.  <Trial  #1> 

9.  Ok,  step.  Rest  a  moment,  then  you  will  do  four  more. 

10.  Ok,  start. 


11.  <Trial  ft 2> 


ID  >*> 


12.  Ck,  stop, 
our  to  eater 
xperiment . 


Rest  a  moment,  this  will  he  ycur  last  set  cf 
for  the  unbuffered  rode  pert  of  the 


13.  Ck,  start. 


14.  <Trial  #3> 

15.  Stop.  You  deserve  a  break.  Relax  a  while.  You  may 
get  up  and  move  around,  get  a  drink,  etc. 
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VOICE-BUFFERED  N'CDE 


1.  Turing  this  portion  of  the  experiment  you  will  view  12 
cards,  and  use  tne  T £00  in  buffered  mode  tc  write  a  report 
for  each  card  like  the  one  you  saw  in  the  sample  (or  other 
part  of  the  exper-  iment).  I  will  step  you  after  every  four 
cards.  This  will  give  you  a  break  and  allow  me  to  collect 
some  data. 

2.  The  T600  buffered  mode  allows  you  to  speak  a  chain  of 
phrases  prior  to  sending  them  tc  tne  nest  computer.  Feu  majf 
edit  the  last  utterance  in  the  buffer  by  saying  "kill  line’ 
or  its  equivalent  for  your  vocabulary.  If  you  make  several 
errors,  the  entire  buffer  nay  be  erased  with  the  command 
"kill  line."  Once  jrou  are  ready  to  send  the  contents  of  the 
buffer,  you  say  "go  ‘  or  "carriage  return,"  whichever  you 
trained,  and  the  character  string  will  be  sent  tc  the  text 
editor  at  ISIE.  Eowever,  you  will  not  be  able  to  use  the 
editing  features  of  the  text  editor  at  ISIE  while  in  the 
buffered  mode.  I  will  demonstrate  the  buffered  mode  for  you 
now  . 

3.  If  you  look  in  the  viewport  at  this  time,  you  will  see 
that  the  three  bottom  lines  of  tne  T600  display  may  be  seen. 
These  will  provide  a  visual  feedback  of  the  buffer  contents, 
and  allow  you  to  view  the  editing  process  as  well  as  the 
card . 

4.  Now  practice  using  the  sample  carl  provided. 

5.  <critique  the  report> 

6.  You  are  to  go  as  fast  as  you  can  while  trying  to 
minimize  errors.  Seep  in  mind  you  are  writing  an 
intelligence  report  which  should  be  timely,  accurate,  and 
complete.  Cuestions? 

7.  Ok,  start. 

6.  ^Trial  *1> 

9.  Ck,  stop.  Rest  a  moment,  then  you  will  do  four  more. 

I?.  Ok,  start. 

11.  CTrial  #2> 
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12.  Ck,  stop.  Rest  a  rorent,  this  will  he 
four  tc  enter  for  the  buffered  •’•ode  part  of 

13.  Ck,  start. 

14.  <Trial  *3. 

15.  Stop.  You  deserve  a  break.  Relax  < 
get  up  and  r-.ove  around,  get  a  drink,  etc. 


your  last  set  cf 
tne  experinent. 


while.  You  rray 
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VOCABULARY  WORDS  MISRECOGN IZED  OR  REJECTED 

NOTE:  THE  FOLLOWING  LIST  IS  IN  ASCENDING  COLLATING  SICUENC 
BY  UTTERANCE  AND  MISRZCOGNITON .  THE  MISRECOGNITIONS  HAV 
THE  FOLLOWING  FORMAT: 

A  (B)  X  N 

WHERE 

A  =  UTTERANCE  ASSOCIATED  WITH  T600 
MISRECOGNITION 

E  =  SPECIFIC  T600  OUTPUT,  IF  DIFFERENT 
THAN  A  ABOVE;  E.G.'\2)"  MEANS 
THAT  A  NUMERAL  WAS  OUTPUT  RATHER 
TEAM  THE  WORD  ’TWO" 

N  =  NUMBER  OF  OCCURENCES 

Xsjisjjcjj:#*:#:*#*###  #  #  j)e  #  Jfc  #*))<#  jj<  s*  s(c  5*  afe  **#*#*## 


*  UTTERANCE  *  *  MI SRECOGN ITI ON ( S  )  * 


*  #  3jt  #  #  #  $  jfc  *  $  *  m  jje 

*  3(e  #####  #  a*  ###*  #  #  lie##  *  jjt  !jc 

122-MM 

120— MM  X  2 

122-N.M 

152-NiMi  X  3 

152-MM 

122-MM  X  £ 

8E-MM 

57-MM 

AA  GUNS 

AN -8  CAMP  X  S 

AA  GUNS 

ANTI-TANK  GUN'S  X  3 

AA  GUNS 

YAK-2SP  FIREBAR 

AIRCRAFT 

ANTI-TANK  GUNS 

AIRCRAFT 

CARRIER 

AIRCRAFT 

TU-26  BACKFIRE 

AMPHIBIOUS 

FRIGATES  X  3 

AN-6  CAMP 

AA  GUNS 

ANTI-TANK  GUNS 

AMPHIBIOUS 

ANTI-TANK  GUNS 

AN -8  CAMP  X  4 

ANTI-TANK  GUNS 

BEEP*  X  £ 

ASSAULT  GUNS 

BEEP*  X  16 

ASSAULT  GUNS 

MISSILE 

ASU-57 

AT-3  SAGGER 

AT-1  SNAPPER 

BEEP* 

AT-2  SAGGER 

APC 

BM-21 

M— 44 

BM-24 

BM-21 

BMP-75PB 

BTR-60PK 

BOATS 

BEEP* 

M  M 


s*  $  #  aje  #  *e  $  #  #  **  #  # 

*  UTTERANCE  * 

#  #  £  *  £  a*  jfc  #  £  *  *  # 


BOMBERS 

BOMBERS 

ECMEERS 

BOMEERS 

ERAVO 

BRAVO  CLASS 

BRAVO  CLASS 

BRAVO  CLASS 

BRDM 

ERDM 

ETR-1E2 

BTR-52PK 

BTR-50PK 

BTR-62PK 

CARRIAGE  RETURN 

CARRIAGE  RETURN 

CARRIAGE  RETURN 

CARRIAGE  RETURN 

CARRIAGE  RETURN 

CARRIAGE  RETURN 

CARRIAGE  RETURN 

CARRIAGE  RETURN 

CARRIAGZ  RETURN 

CARRIAGE  RETURN 

CARP  I AGE  RETURN 

CARRIAGE  RETURN 

CHARLIE  I  CLASS 

CHARLIE  I  CLASS 

CHARLIE  I  CLASS 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 


#  #  ajc  #  a*  ajt  a*  ax  *  »!<  #  #  a*  *  ajs  if  $  Ijt  #  a*  aje 

*  MISRZCGGNITICN (S)  * 

aje  aje  at:####  i(e  *«  **  ajs  a*  aje  aje  a(c  aj:  aje  aje  $  ##  # 


ALPHA  l A } 

BEEF*  X  7 
IL-14  CRATE  X  2 
LAUNCEERS 
FRIGATE 
GOLF  I  CLASS 
KOMAR  CLASS  X  2 
KOTLIN  CLASS  X  2 
TORPEDO 
YANKEE 

NINETEEN  (19) 
BTR-60PK  X  4 
D-20 

BTR-EePt  X  3 
AN -6  CAM.?  X  4 
BEEP*  X  2 
BRDM 

CARRIER  X  11 
FRIGATE  X  2 
FRIGATES 
HEAVY  X  3 
SSN 

VICTOR  (V) 

XRAY  (X) 

YAK-26P  FIRFEAR  X  2 

ZSU-23/4 

FOXTROT  CIASS 

KOTLIN  CLASS 

MIRKA  I  CLASS  X  2 

AIRBORNE 

BEEP*  X  148 

BOMEERS 

3RAVC  IB)  X  4 

BRDM 

ELEVEN  (11)  X  6 
FIVE  (5)  X  7 
FOUR  (4) 

HEAVY  X  4 
KOTLIN  CLASS 
LANDING 
LIMA  (L)  X  l 
MI-4  HOUND  X  5 
MIKE  (M)  X  2 
NINE  (9) 

NOVEMBER  (N)  X  2 
SA-8  GECKO 
SEVEN  (7)  X  2 


5jt  »C* *£ * 

*  UTTERANCE  * 

*  *s  >jt *s  #  *c  $  #  sjt  *s  Jit  *s  *s 


CCNFI RMED 

CONFIRMED 

CONFIRMEE 

CONFIRMEE 

CONFIRMEE 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CONFIRMED 

CRUISERS 

D-30 

E-44 

EASH 

EASE 

DELETE  LINE 

DELETE  WORD 

DELETE  WORD 

DELTA  CLASS 

DELTA  II  CLASS 

ECEO  I  CLASS 

ECHO  II  CLASS 

ECHO  II  CLASS 

ECEC  II  CLASS 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

EIGHT 

ELEVEN 

ELEVEN 

ELEVEN 

ELEVEN 

ELEVEN 

ELEVEN 

ELEVEN 

ERASE 

FIELD  GUNS 
HELD  GUNS 


# # #  * *  Sjt  #  SjC *  $  sit  sjtsfc##*# *C 

*  MISRECCC-N 1TIC-N  ( S  )  * 

v##  ####*>!«  *#####  ###*£## 


TEN  (12) 

TWELVE  l IE )  X  ? 
TWENTY  (22) 
TWENTY-FIVE  (25) 
TWENTY-ONE  (21) 
UNIFORM  (U) 

UPPER  RIGHT 
XRAY  (X)  X  e 

7  C  TT_ O'*  /A 

TWENTY-THREE  (23) 
D-74 

TWENTY-FOUR  (24) 
QUEBEC  (Q) 

TEN  (12) 

LIMA  (L) 

DELETE  LINE  (CTRL  X) 
TWENTY-THREE  (23) 
K.CTLIN  CLASS  X  2 
GOLF  II  CLASS  X  2 
PETYA  I  CLASS  X  2 
DELTA  II  CLASS  X  2 
PETYA  II  CLASS 
SHERSEEN  CLASS 
AA  GUNS  X  4 
AMPHIBIOUS 
AN-8  CAMP  X  3 
APC 

ASU-65 
BEEP*  X  4 
EIGHTEEN  (18)  X  4 
FIFTEEN  (15) 

FOUR  (4)  X  5 
HEAVY  X  6 
KA-15  HEN  X  14 
MEDIUM 
SA-8  GECKO 
YANKEE  (Y)  X  7 
BEEP*  X  8 
D-20 

FIVE  (5)  X  2 
FOUR  (4) 

ONE  (1)  X  3 
UPPER  LEFT 
UPPER  RIGHT 
EIGHT  (8) 

BEEP*  X  2 
JULIETT  X  2 
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sue*#### 

*  UTTERANCE  * 
*$##***$#***# 


EIEIE  GUNS 
FIELD  HOWITZERS 
FIELD  HOWITZERS 
FIELD  HOWITZERS 
FIFTEEN 
FIFTEEN 
FIGHTER 

FIGHTER-BOMBERS 

FIGHTER-BOBBERS 

FIGHTER-BOMBERS 

FIVE 

FIVE 

FIVE 

FIVE 

FIVE 

FORTY 

FOUR 

FOUR 

FOUR 

FOUR 

FRIGATE 

FRIGATES 

FRIGATES 

FRIGATES 

FROG-3 

FROG -3 

FROG -4 

GO 

GO 

GO 

GO 

GO 

GO 

GO 

GOLF  I  CLASS 
GRISHA  CLASS 
GRISHA  CLASS 
GRISHA  CLASS 
HEAVY 

HELICOPTERS 
HELICOPTERS 
HELICOPTERS 
HELICOPTERS 
HOTEL  III  CLASS 
IL-14  CRATE 
INSTALLATION 
INSTALLATION 


#  *  *  #  #  #  #  #  #  #  *  *  #  jX  aje  #  $  *  *  *c  #  # 

*  MISRECCGNITICN(S)  * 
*##*#**####*##*#*#**>!«# 


T  — 1  7 

BEEP*  X  5 
HELICOPTERS  X  4 
INTELLIGENCE 
EIGHTEEN  (18) 
THIRTEEN  (13)  X  5 
FRIGATES 

ROCKET  LAUNCEERS 
BEEP*  X  3 
TWENTY-ONE  (21) 
AN-8  CAMP 
BEEP*  X  2 
NINE  (9)  X  2 
PAPA  (P)  X  2 
QUEBEC  (Q)  X  2 
THREE  (3) 

BEEP*  X  33 
FROG -4 

LOWER  RIGHT  X  4 
MOBILE 
IL-38  MAY 
BEEP* 

FRIGATE 

SHSRSHEN  CLASS 
BEEP* 

D-20 

PROBABLE 
BEEP*  X  24 
BRAVO  (B)  X  2 
DELTA  (D) 

ECHO  (E)  X  3 
GOLF  (G) 

TWELVE  (12) 

ZERO  (0) 

OSA  I  CLASS 
XYNDA  CLASS 
RIGA  CLASS 
VICTOR  CLASS 
SCUD  B 
BEEP*  X  t 
BRAVO  (B) 

FOXTROT  (?) 

MI-4  HOUND 
HOTEL  II  CLASS  X  3 
MI-24  HIND 
BEEP*  X  3 
S-60 
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*  UTTERANCE  * 


INTELLIGENCE 
INTERCEPTORS 
INTERCEPTORS 
JULIET  CLASS 
KA-18  HOG 
KANIN  CLASS 
KANIN  CLASS 
KANIN  CLASS 
KANIN  CLASS 
KANIN  CLASS 
KANIN  CLASS 
KARA  CLASS 
KARA  CLASS 
KARA  CLASS 
KARA  CLASS 
KARA  CLASS 
KASHIN  CLASS 
KASHIN  CLASS 
KASHIN  CLASS 
KASHIN  CLASS 
KASHIN  CLASS 
KASHIN  CLASS 
KIEV  CLASS 
KIEV  CLASS 
KIEV  CLASS 
KIEV  CLASS 
KIEV  CLASS 
KIEV  CLASS 
KIEV  CLASS 
KILL  LINE 
KILL  LINE 
KILL  LINE 
KILL  LINE 
KILL  LINE 
KILL  LINE 
KILL  WORD 
KILL  WORD 
KILL  WORD 
KILL  WORE 
KOMAR  CLASS 
KOMAR  CLASS 
KOTLIN  CLASS 
KOTLIN  CLASS 
KOTLIN  CLASS 
KOTLIN  CLASS 
KOTLIN  CLASS 
KOTLIN  CLASS 


### #  *e  $ ### #  >jr#  * ##$### # *  if 

*  MI SRSCOGN ITICN ( S  )  * 

###$**#*##  s)t  #*###  * 


BEEP*  X  e 
BEEP*  X  3 
HELICOPTERS  X  6 
YURKA  CLASS 
EIGHT  (8) 

CARRIER 

KASHIN  CLASS  X  3 
KIEV  CLASS 
XYNDA  CLASS 
SHERSHEN  CLASS  X  4 
YANKEE  CLASS  X  5 
KANIN  CLASS 
KOMAR  CLASS 
KOTLIN  CLASS  X  3 
STENKA  CLASS 
YURKA  CLASS  X  2 
JULIET  CLASS  X  8 
KANIN  CLASS 
KOTLIN  CLASS 
NATYA  CLASS  X  3 
SHERSESN  CLASS  X  2 
YANKEE  CLASS 
AIRCRAFT 
JULIET  CLASS 
KANIN  CLASS 
KARA  CLASS 
KYNDA  CLASS  X  2 
SHERSHEN  CLASS 
STENKA  CLASS  X  6 
CHARLIE  lC)  X  6 
DELETE  (CTRL  X) 
KANIN  CLASS  X  2 
KOTLIN  CLASS 
M-44 

MI -4  HOUND 
BEEP*  X  8 
FIELD  HOWITZERS 
KILL  LINE  X  2 
SEVEN  (7) 

KARA  CLASS  X  2 
MIRKA  I  CLASS 
BRAVO  CLASS 
CHARLIE  II  CLASS 
DELTA  CLASS 
KASHIN  CLASS 
KCMAR  CLASS  X  2 
MOSKVA  CLASS 
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#  #  #  « Sjc  *  *  #  Jjc *  #  #  $ 

*  UTTERANCE  * 

St  JCC  ##*£*##  Xcjjcajejjtjjs 

KOTLIN  CLASS 
KOTLIN  CLASS 
KOTLIN  CLASS 
SRESTA  I  CLASS 
KRSSTA  I  CLASS 
KRISTA  I  CLASS 
KRISTA  I  CLASS 
KRESTA  II  CLASS 
KRISTA  II  CLASS 
KRIVAK  CLASS 
KRIVAK  CLASS 
KRIVAK  CLASS 
KRIVAK  CLASS 
KYNDA  CLASS 
KYNDA  CLASS 
KYNDA  CLASS 
KYNDA  CLASS 
LANDING 
LAUNCHERS 
LIGHT 
LIGET 
LIGHT 
LIGHT 
LIGHT 
LIGHT 

LOWE?  LEFT 
LOWER  LEFT 
LOWER  LEFT 
LOWER  LEFT 
LOWER  LEFT 
LOWER  LEFT 
LOWER  LEFT 
LOWER  LEFT 
LOWER  LEFT 
LOWER  LEFT 
LOWER  RIGHT 
LOWER  RIGHT 
LOWER  RIGHT 
LOWER  RIGHT 
LOWER  RIGHT 
LOWER  RIGHT 
LOWER  RIGHT 
LOWER  RIGHT 
M-1955 
M-1S55 
M-1955 
Mi-1976 


St#*#*#* 

*  MISRECCGNITICN(S)  * 

St########***###*#*#*** 


POLNOCNY  CLASS 
POT  I  CLASS 
UPPER  LEFT 
CHARLIE  I  CLASS 
ECEO  I  CLASS 
OSA  I  CLASS  X  4 
RIGA  CLASS 
NANUCEXA  CLASS 
OSA  II  CLASS 
KANIN  CLASS 
KARA  CLASS  X  2 
KOTLIN  CLASS 
KYNDA  CLASS 
KANIN  CLASS  X  8 
NATYA  CLASS 
RIGA  CLASS 
STENKA  CLASS  X  7 
REEF* 

MORTARS  X  6 
BEEP*  X  2 
FIVE  (5) 

LOWER  RIGHT  X  2 
MIKE  (H )  X  4 
ONE  (1) 

TWENTY  (2?)  X  3 
BEEP*  X  19 
BTR-SZPK 
CORVETTES  X  2 
KOTLIN  CLASS 
LIGHT 

LOWER  RIGHT  X  9 
MOBILE  X  3 
THREE  (3) 
TWENTY-ONE  (21) 
UPPER  LEFT  X  4 
BEEP* 

CONFIRMED 

LIGET 

LOWER  LEFT 
ONE  (1)  X  3 
SEVEN  (7) 
SEVENTEEN  (17) 
UPPER  RIGHT  X  8 
ASU-efc 
BEEP*  X  2 
SU-15  FLAGON 
PT-76 
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•#  #  #  *  *  $  5*  *  *  #  *  #  # 

*  UTTERANCE  * 


M-240 
M-4  BISON 
iv ~4  EISON 
M-4  BISON 
.*-44 
M— 49 
M-49 
MEDIUM 
M.I-10  HARKS 
MI-10  HARKE 
MI-12  HOMER 
MIG-19  FARMER 
MIG-21  FISHBIL 
MIG-21  FISHBID 
MIG-23  FLOGGSR 
MIG-25  FOXBAT 
MIG-25R  FOXBAT 
MIG-25R  FOXBAT 
MIRK A  I  CLASS 
MIRKA  I  CLASS 
M IRKA  II  CLASS 
MIRKA  II  CLASS 
MIRKA  II  CLASS 
MIRKA  II  CLASS 
MIRKA  II  CLASS 
MISSILE 
MOBILE 
MOBILE 
MOBILE 
MOBILE 
MORTARS 
MOSKVA  CLASS 
MOSKVA  CLASS 
MOSKVA  CLASS 
MOSKVA  CLASS 
NANUCHKA  CLASS 
NANUCEKA  CLASS 
NANUCHKA  CLASS 
NANUCHKA  CLASS 
NANUCEKA  CLASS 
NANUCHKA  CLASS 
NATYA  CLASS 
NATYA  CLASS 
NATYA  CLASS 
NATYA  CLASS 
NATYA  CLASS 
NATYA  CLASS 


*  *  *  #  #  sjt  #  sjs  #  *  x  #  #  #  S*  >•:  #  *  *s  #  « 

*  MISRECOGN ITION ( S )  * 


BEEP* 

BEEP* 

CHARLIE  I  CLASS 
MI-6  EOOK 

TWENTY-FOUR  (24)  X  3 
M-1955 

TWENTY-FIVE  (25) 
BEEP* 

MI-24  HIND  X  3 
MI-6  EIP 

mig-is  farmer 
RIGA  CLASS 
IL-14  CRATE 
MIG-2?  ELCGGER 
KA-25  HORMONE 
MIG-25R  FOXBAT  X  2 
KA-25  HORMONE 
MIG-25  FOXBAT  X  3 
ECHO  II  CLASS 
PETYA  I  CLASS 
CHARLIE  II  CLASS  X  2 
DELTA  II  CLASS 
KANIN  CLASS 
KOTLIN  CLASS  X  2 
PCLNOCNY  CLASS 
TWELVE  (12) 

BEEP*  X  3 
ERAVO  X  2 
HOTEL  (H) 

PROBABLE  X  6 
LAUNCEERS 
BEEP*  X  2 
GOLF  I  CLASS  X  3 
NATYA  CLASS 
PCLNOCNY  CLASS  X  5 
KOTLIN-SAM  CLASS 
KYNEA  CLASS 
SHERSHEN  CLASS 
STENKA  CLASS  X  2 
YANKEE  CLASS 
YURKA  CLASS 
ALLIGATOR  CLASS  X  2 
BEEP* 

KANIN  CLASS  X  3 
KASHIN  CLASS  X  2 
KOTLIN  CLASS 
KYNDA  CLASS 


141 


*  UTTERANCE  *  *  MI SRECCGN ITICN (S )  * 


it  >jc  #  xn  $  *##*###  Kt 


###j}ej!e:fri)cs(cjjcjjc#>*j)cj*5it#»‘<;fc>)c>)sj',t# 


NATYA  CLASS 

NINE 

NINE 

NINE 

NINE 

NINE 

NINE 

NINETEEN 

NINETEEN 

NINETEEN 

ONE 

ONE 

CNE 

ONE 

CNE 

ONE 

CNE 

CS A  I  CLASS 
CSA  II  CLASS 
FETYA  I  CLASS 
PETYA  I  CLASS 
PETYA  II  CLASS 
PETYA  II  CLASS 
FETYA  II  CLASS 
PETYA  II  CLASS 
POLNOCNY  CLASS 
POLNOCNY  CLASS 
POLNCCNY  CLASS 
POLNOCNY  CLASS 
POLNOCNY  CLASS 
POLNCCNY  CLASS 
POLNOCNY  CLASS 
POSSIBLE 
POTI  CLASS 
POT I  CLASS 
POTI  CLASS 
POTI  CLASS 
POTI  CLASS 
PRIMORYE  CLASS 
PRIMORYE  CLASS 
PRIMORYE  CLASS 
FROBAELE 
PROBABLE 
PROBABLE 
PROBABLE 
PROBABLE 
PROBALBLS 


POTI  CLASS 
BEEP*  X  2 
FIVE  (5)  X  5 
LIGHT 
MI -6  HIP 
MIKE  (M) 

TWENTY  (20)  X  6 
EIGHTEEN 
MIKE  (M) 

THIRTEEN  (13) 

BEEP*  X  4 
EIVE  (5)  -X  7 
POUR  (4) 

FOURTEEN  (14)  X  2 

LIGHT  X  2 

M-44 

UPPER  RIGHT 
MIRKA  I  CLASS 
KRESTA  II  CLASS  X  2 
NANUCHXA  CLASS 
YANKEE  CLASS 
ECHO  II  CLASS 
HOTEL  II  CLASS 
KASHIN  CLASS 
SHERSEEN  CLASS 
ALLIGATOR  CLASS 
BEEP*  X  2 
ECHO  II  CLASS 
HOTEL  II  CLASS 
HOTEL  III  CLASS 
KOTLIN  CLASS 
MOSKVA  CLASS 
BEEP* 

KANIN  CLASS 
KOTLIN  CLASS  X  3 
MOSKVA  CLASS  X  6 
ROMEO  CLASS 
WHISKEY  CLASS  X  2 
ECHO  I  CLASS 
MIRKA  I  CLASS 
MIRKA  II  CLASS 
BEEP* 

BRAVO  (B)  X  7 
MOBILE  X  3 
POSSIBLE 
TORPEDO 

TWENTY-FOUR  (24) 
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*  UTTERANCE  * 


RECONNAISSANCE 
RECONNAISSANCE 
RECONNAISSANCE 
RECONNAISSANCE 
REPEAT  LINE 
REPEAT  LINE 
REPEAT  LINE 
REPEAT  LINE 
RETURN 
RETURN 
RETURN- 
RETURN 
RETURN 
RIGA  CLASS 
RIGA  CLASS 
ROMEO  CLASS 

s-ee 

S-60 

SEVEN 

SEVEN 

SEVEN 

SEVEN 

SEVEN 

SEVEN 

SEVEN 

SEVEN 

SEVEN 

SEVENTEEN 

SEERSESN  CLASS 

SHIPS 

SIX 

SIX 

SIX 

SIX 

SIX 

SIX 

SIX 

SIX 

SIXTEEN 

SIXTEEN 

SPACE 

SPACE 

SPACE 

SPACE 

SPACE 

SS 

SS 


#  $  #  #  #  $  #  $  #  $  ##  #  V  Jjt  V  *  #  #  *  # * 

*  MIS RE CO GN  IT  I  ON ( S  )  * 


BEEP*  X  4 

CRUISERS 

GRISHA  CLASS 

INTELLIGENCE  X  2 

CARRIAGE  RETURN  (CTRL  M ) 

D-20 

M-240 

TERSE  1 3 ) 

BEEF*  X  4 
CONFIRMED  X  5 

eleven  in; 

SEVEN  (7)  X  2 
TEN  (10)  X  2 
GRISHA  CLASS  X  £ 

VICTOR  CLASS  X  3 
POLNOCNY  CLASS 
SS-16 
SSG  X  2 
ASSAULT  GUNS 
BEEP*  X  3 
ELEVEN 
FIVE  (5) 

SCUD  A  X  11 
SEVENTEEN  (17) 

SISRA  (S)  X  2 
WHISKEY  CLASS 
ZSU-57/2 
SCUD  A  X  3 
KANIN  CLASS  X  4 
SIX  (6)  X  2 
BEEP* 

DESTROYERS  X  3 
FRIGATES 
INDIA  (I) 

SCUD  B 
SHIPS  X  9 
SPACE  (  )  X  20 
T-72 
BEEP* 

FIFTEEN  (15)  X  4 

AMPHIBIOUS  X3 

BACKSPACE  (CTRL  A) 

FRIGATES  X  2 

SHIPS 

T-1Z 

SSGN 

SSM 
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*  jjcjjc#j)t>*j)cjfr  jjtjjtjjttjejjc 

*  UTTERANCE  * 

*  a*#  *  #  $  jjt  #  j*  jjt  jjs  gj<  jjt 


$ *« Jjt  #  *  jjc  #  jjc  #  Jjc  Jjc  >jt  sis  Jjt  Jjc  #  jjc jjc  #  jjt  jjt jjc 

*  MISRECOGNITION(S)  * 

jXJpjjcjjtjjcjjcJjcjjcJjtjjc^cjjcjjtjjcKeStjjijjcj^jjcjfcjjc 


SS-14  SCAPEGOAT 

SA-6  GECKO  X  3 

SS-20 

SSE 

SSB 

SSG  X  6 

SSBN 

SSB  X  5 

SSBK 

SSGN  X  12 

SSG 

SSE  X  2 

SSG 

SSGN 

SSGN 

CHARLIE  I  CLASS 

SSGN 

SSEN  X  31 

SSGN 

SSG 

SSGN 

SSM 

SSGN 

SSN  X  4 

SSM 

SA-6  GAINFUL 

ssm 

SSN  X  22 

SSN 

SSM 

STENKA  CLASS 

JULIET  CLASS 

STENKA  CLASS 

KANIN  CLASS 

STENKA  CLASS 

KTNDA  CLASS 

STENKA  CLASS 

NATYA  CLASS  X  2 

STENKA  CLASS 

SEERSHEN  CLASS 

STENKA  CLASS 

VICTOR  CLASS  X  2 

STRIKE/ATTACK 

AN -6  CAMP 

STRIKE/ATTACK 

BEEP*  X  2 

STRIKS/ATTACK 

M-1955 

SU-15  FLAGON 

IL-3&  MAY 

SU-19  FENCER 

SS-16 

SU-A9  FISHPOT 

3EEP* 

SVERDLOV 

BEEP* 

SVERDLOV  CLASS 

OSA  I  CLASS  X  2 

SVERDLOV  CLASS 

POLNCCNY  CLASS 

SVERDLOV  CLASS 

STENKA  CLASS  X  2 

SVERDLOV  CLASS 

YUF.KA  II  CLASS 

T-34/65 

ASU-85 

T-34/S5 

M-1955  X  2 

T-34/55 

MIG-21  FISEBED 

T-34/85 

T-54/55 

T-34/S5 

TU-26  BACKFIRE  X  3 

T-54/55 

BEEP*  X  2 

T-54/55 

BMP-76PB 

T-54/55 

D-44 

T-54/55 

T-34/85  X  22 

T-S2 

TU-15  BADGER 

TANGO 

NINE  (9) 

TANGO  CLASS 

NATYA  CLASS 

TANKS 

BEEP* 

TANKS 

BEEP*  X  2 

TANKS 

HEAVY 

*  ITERANCE  * 


if  $  $  ttf  $  #  #  $###  #  *  #  if.  $  >jc  s(c 

*  MISRSCCGNITION(S)  * 

if  Xf  »Jc^e #  8##  8  #88  8 8  #888888  8 


TANKS 

NINETEEN  (19) 

TANKS 

TWENTY  (20) 

TEN 

BEEP* 

TEN 

ELEVEN  (11) 

THIRTEEN 

FIFTEEN  (15) 

THREE 

CARRIER 

THREE 

FOURTEEN  (14) 

X  2 

THREE 

FRIGATE  X  8 

THREE 

FROG -3 

THREE 

HEAVY  X  2 

THREE 

Mi-e  hip 

THREE 

THIRTEEN  (12) 

THREE 

TWENTY  (22) 

THREE 

TWO  (2) 

TEREE 

WHISKEY  (W) 

TU-22  ELINDER 

TU-22  BEAR 

TU-2S  BEAGLE 

IL-14  CRATE  X 

2 

TU-2SP  FIDDLER 

MIG-23  FLCGGER 

TWELVE 

BEEP* 

TWELVE 

GOLF 

TWENTY 

BEEF*  X  4 

TWENTY 

D-30 

TWENTY 

FOURTEEN  (14; 

TWENTY 

LIGHT 

TWENTY 

MIKE  (M) 

TWENTY 

NINETEEN  (19) 

TWENTY 

ONE  (1) 

TWENTY 

UPPER  RIGHT 

TWENTY-FOUR 

TWENTY-CNE  (21) 

TWENTY-ONE 

BEEP*  X  2 

TWENTY-ONE 

TWENTY-FOUR  (24)  X  2 

TWENTY-THREE 

FROG-3 

TWENTY-TWO 

BEEP*  X  2 

LOWER  LEFT 

LOWER  RIGHT  X 

9 

TWENTY-TWO 

TWENTY-THREE 

TWO 

BEEP*  X  12 

TWO 

BTR-1E2 

TWO 

EIGHT  (6) 

TWO 

FIFTEEN  (15) 

TWO 

FOUR  (4)  X  2 

TWO 

FRIGATE 

TWO 

HEAVY 

TWO 

LIGHT 

TWO 

MEDIUM  X  4 

TWO 

T-10 

TWO 

T-62 

TWO 

T-72 
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sSsfcsjs*##  Jjc  #*###« 

*  UTTERANCE  * 

TWC 

TWO 

TWO 

TWC 

TWO 

TWO 

TWO 

UNIFORM 
UPPER  LEFT 
UPPER  LEFT 
UPPER  LEFT 
UPPER  LEFT 
UPPER  LEFT 
UPPER  LEFT 
UPPER  LEFT 
UPPER  RIGHT 
UPPER  RIGHT 
UPPER  RIGHT 
UPPER  RIGHT 
UPPER  RIGHT 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 
VICTOR 

VICTOR  CLASS 
VICTOR  CLASS 
VICTOR  CLASS 
VICTOR  CLASS 
VICTOR  CLASS 
WHISKEY  CLASS 
YAK-26P  FIREBAR 
YA2-26P  FIREBAR 
YANKEE  CLASS 
YURKA  CLASS 
YURKA  CLASS 
YURKA  CLASS 


#  #  #  #  *  $  tit  #  #  $  if  #  #  *  #  Jjs  aOc  *  #  s*  * 

*  MISRSCCGN  ITION ( S )  * 

Xc  #  *«  #  aje  $  #  *e  #  afs #  #  $  *  *  sic 


TEN  (12)  X  4 
THREE  ^31  X  £ 

TU-22  BLINDER 
TWENTY-TWO  (22)  X  5 
YAK-26?  FIREBAR 
ZSU-57/2  X  12 
ZU-23/2  X  6 
BREM 

BEEP*  X  21 
BRAVO  CLASS  X  3 
EIGHTEEN  (16) 

ELEVEN  (11) 

KOTLIN  CLASS  X  3 
LOWER  LEFT  X  5 
UPPER  RIGHT  X  5 
EEEP*  X  6 
LIGHT 

LOWER  LEFT  X  4 
LOWER  RIGHT  X  9 
UPPER  LEFT  X  7 
BEEP*  X  17 
CARRIAGE  RETURN 
CARRIER  X  2 
D-30 

FIFTEEN  (15)  X  2 
FRIGATE  X  2 
HEAVY 

INDIA  (I)  X  2 

M-1  QC,5 

NOVEMBER  ( N ) 

QUEBEC  (Q)  X  2 
SA-6  GECKO 
THREE  (3)  X  2 
TU-2e  BEAR 
WHISKEY 
BEEP* 

KARA  CLASS 
KYNEA  CLASS  X  2 
MIP.KA  II  CLASS 
NANUCHKA  CLASS 
KANIN  CLASS 
MIG-19  FARMER 
TU-28P  FIDDLER 
KANIN  CLASS  X  4 
KYNDA  CLASS 
PR1MORYE  CLASS 
VICTOR  CLASS  X  3 
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*  KISB3C0GNITI0N ( S  5  * 

#  ax  #  *  #  #  #  *  #  *t  3jc  #  *c  ajt  Jit  #  *  #  if  # »X  >jt 


Jje  9  aje  KS  #  *  #  #  Jji #  *s  #  # 

*  UTTERANCE  * 

a*#*########## 


ZERO 

ZERO 

ZERO 

ZERO 

ZSU-23/4 

ZSU-57/2 

ZSU-57/2 

ZSU-57/2 

ZU-23/2 

ZU-23/2 

ZU-23/2 


BACKSPACE  (CTRL  A) 
KILO  (K)  X  2 
hOEILE 

ZSU-57/2  X  2 
ZU-23/2  X  2 
ASU-57 
ZSU-23/4 
ZU-23/2 
SU-19  FENCER 
TU-22  BLINDER 
TU-26  BACKFIRE 


APPENDIX  I 


RESULTS  FOR  PP.E/POST  SUBJECTIVE  QUESTIONNAIRE 

The  following  data  reflect  whether  subjects'  attitudes 
shifted  either  toward  typing;  or  vcice  as  a  result  cf  the 
experiment.  A  two-tailed  sign  test  was  used.  Note:  Means 
for  the  pre/post  given  below  may  be  misleading  if  thought  to 
be  indicative  of  the  shift.  The  sign  test  loots  at  the  fact 
of  whether  their  was  a  shift  or  not,  and  ignores  the  amount 
of  shift  in  the  analysis,  since  the  amount  may  be  somewhat 
arbitrary . 


QUESTIONS  and 
PRE  /  POST  MEANS 
for  20  subjects. 


1.  Which  data  entry  mode 
do  you  think  is  the 
easiest  to  use  to  enter 
character  strings  and 
commands?  ^5. 1/6.1) 

2.  Which  data  entry  mode 
do  ycu  think  is  the 
fastest  mode  for  entering 
character  strings  and 
commands?  (5.1/5. 6} 

3.  Which  data  entry  mode 
is  the  most  accurate  for 
entering  character 
strings  and  commands? 

(4. 1/4. 8) 

4.  Which  data  entry  mode 
provides  the  most 
flexibility,  in  general, 
for  interaction  with  a 
computer?  (5. 1/5.1) 

5.  Which  data  entry  mode 
would  you  prefer  tc 
operate  for  several 
hours,  if  required? 

(4. 3/4. 3) 


SHIFTS 

toward 

TYPING 

SHIFTS 
towa  rd 
VOICE 

NO 

SHIFT 

ac  =  .10 

Signif? 

V 

12 

c; 

YES 

3 

13 

4 

YES 

4 

9 

? 

NO 

4 

12 

4 

YES 

3 

8 

9 

NO 

QUESTIONS  and 

PRE  /  POST  MEANS 
for  20  subjects. 

SHIFTS 

toward 

TYPING 

SHIFTS 

toward 

VOICE 

NO 

SHIFT 

oc  =.10 
Signif? 

6.  Which  data  entry  rrode 
would  you  prefer  to 
operate  as  a  more 
sporadic  user  o?  a 
computer  system? 

(4. 2/4. 3) 

'Z 

w 

10 

7 

YES 

7.  Which  data  entry  mode 
promotes  the  most  relaxed 
operation?  (5. 0,5.1) 

£ 

6 

7 

NO 

9.  Which  data  entry  mcie 
would  be  the  most 
advantageous  to  use  to 
update  an  on-line  data 
base  of  intelligence 
information?  (5. 1/5.0) 

3 

9 

s 

NO 

9.  Which  data  entry  mode 
provides  the  best 
man-machine  interface  in 
a  time-critical,  high 
pressure  work 
environment?  (£.0/5.0) 

5 

12 

w> 

NC 

10.  Which  data  entry 
mode  do  you  think  is  the 
easiest  to  to  learn? 

2 

13 

5 

YES 

14.9/5.6) 
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£7.  Army  Communicative  Technology  Office  1 

ATTN:  Major  W.  MacHarrie 
Eox  4337 

Fort  Sustis,  Virginia  23624 

£S .  National  Securtity  Agency  1 

ATTN:  Charles  Wayne 
RS4 

Fort  Meade,  Maryland  20755 

£9.  DAVAA-3  1 

Avionics  Research  and  Development 
ATTN:  Lockwood  Reed 
Fort  Monmouth,  New  Jersey  27723 

70.  Army  Research  Institute  1 

PERI-0U 

ATTN:  J.  N.  McConnell 
5201  Eisenhower  Avenue 
Alexandria,  Virginia  22333 

71.  Air  Force  Aerospace  Medical  Research  Lab/BBA  1 

ATTN:  Richard  McKinley 

Wright  Patterson  AFB ,  Chio  45433 
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72.  Texas  Instruments,  Inc. 

ATTN:  George  Doddington 
Box  225936 

MS  371 

Dallas,  Texas  75243 

73.  IBM  Research  Center 
ATTN:  N.  Rex  Dixon 
Box  218 

Torktown  Heights,  New  York  10596 

74.  Massachusetts  Institute  cf  Technology 
ATTN:  Victor  Due 

Room  36~ 543 

Cambridge,  Massachusetts  22139 

75.  Bolt,  Beranek,  and  Newman,  Inc. 

ATTN:  Jared  Wclf 

52  Moulton  Street 

Cambridge,  Massachusetts  02238 

76.  Naval  Air  Development  Center 
ATTN:  Norm  Warner,  Code  6021 
Warminster,  Pennsylvania  18974 

77.  Robert  Lynchard 
3165  McCrory  Place 
Orlande,  Forida  32803 

78.  Naval  Aerospace  Medical  Research  Lab 
ATTN:  Cdr  James  Goodson 

Naval  Air  Station 
Pensacola,  Florida  32508 

79.  Armament  Division/XRC 
ATTN:  H.  I.  Brown 
Eglin  AF3,  Florida  32541 

60.  Naval  Aerospace  Medical  Research  Lab 
Acoustical  Sciences 
ATTN:  Carl  Williams 
Naval  Air  Station 
Pensacola,  Florida  32508 

81.  Naval  Air  Test  Center 

Systems  Engineering  Test  Directorate 
ATTN:  Andrew  Cruce,  Code  57030 
Patuxent  River,  Maryland  20670 
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1 


1 
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£2.  Internal  Revenue  Service 
ATTN:  Thomas  Cullen 
1221  East  Street  NW 
Washington,  D.C.  22224 

63.  Internal  Revenue  Service 
ATTN:  Klaus  Erosius 
1221  East  Street  NW 
Washington,  D.C.  22224 

54.  Vertex  Corporation 
ATTN:  Janet  3aker 
Two  Oak  Park 

Bedford,  Massachusetts  21732 

55.  United  States  Postal  Service 
Research  and  Development  Lab 
ATTN:  Arnold  Craft 

11711  Parklavn  Drive 
Rockville,  Maryland  22852 

£6.  Army  Research  Institute 
ATTN :  Irv  Alderman 
5201  Eisenhower  Avenue 
Alexandria,  Virginia  22333 

£7.  Naval  Ocean  Systems  Center 

ATTN:  John  Phillips,  Code  7232 
San  Diego,  California  S2152 

£8.  Wayne  Lea 

869  Sanford  Court 

Santa  Barbara,  California  93111 

69.  Naval  Electronics  Systems  Center 
ATTN:  Erank  Deckelman,  Code  332 
2511  Jefferson  Davis  Highway 
Arlington,  Virginia  22362 

90.  Naval  Ocean  Systems  Center 
ATTN:  Bruno  White,  Code  6322 
San  Diego,  California  92152 

91.  Naval  Ocean  Systems  Center 
ATTN:  William  Dejka,  Code  e302 
San  Diego,  California  92152 
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92.  Advanced  Research  Projects  Agency/ IPTO 
ATTN:  Lcdr  J.  Bietzler,  USN 
1420  Wilson  Blvd. 

Arlington,  Virginia  2?36£ 


93.  Bolt,  Beranek,  end  Newman,  Inc. 
ATTN:  Richard  Pew 
50  Moulton  Street 
Cambridge,  Massachusetts  20138 


94.  University  of  Missouri  at  Columbia 
Industrial  Engineering  Department 
ATTN:  Marlin  Thomas 

Room  113,  Electrical  Engineering  Building 
Columbia,  Missouri  64211 


S£ .  Office  of  Naval  Research 

ATTN:  M.  Talcctt,  Code  455 
£00  North  Cuincy  Street 
Arlington,  Virginia  22217 

96.  Office  of  Naval  Research 
ATTN:  G.  Malecki,  Code  455 
Arlington,  Virginia  22217 

97.  Bolt,  Beranek,  and  Newman,  Inc. 
ATTN:  N.  Greenfield 

50  Moulton  Street 

Cambridge,  Massachusetts  02136 


98.  Air  Force  Human  Resources  Lab/TT 
ATTN:  Col  Richard  Shelton,  USAF 
Lowry  AFB ,  Colorado  60230 


99.  Armed  Forces  Air  Intelligence  Training  Center 
ATTN:  Capt  Dick  Rewalt 
Building  360 

Lcwry  AFB,  Colorado  60230 


120.  University  of  Southern  California 
Information  Sciences  Institute 
ATTN:  R.  Bisbey 
4676  Admiralty  Way 
Marina  Del  Ray,  California  90291 


101.  Naval  Ocean  Systems  Center 
ATTN:  R.  Kolb,  Code  624 
San  Diego,  California  92152 
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1 


162 


-  r-Mnn  mirier 


102.  Commander  in  Chief  Pacific  Fleet 
ATTN:  Car  R.  Meinhold,  Code  £4 
Bex  5 

Pearl  Earbcr,  Hawaii  96? 60 

103.  Stanford  Research  Institute 
Artificial  Intellignce  Center 
ATTN:  Daniel  Sagalowicz 

33  Ravenswood  Avenue 
Menlo  Park,  California  94025 

104.  Science  Applications  Incorporated 
ATTN:  Russ  Hammond 

Suite  1200 

1911  North  Fort  Meyer  Drive 
Arlington,  Virginia  22209 

105.  S.  Parsons 

19740  Via  Escuela  Drive 
Saratoga,  California  95070 

106.  University  of  Michigan 
Industrial  Engineering  Department 
ATTN:  Don  Chaffin 

Ann  Arbor,  Michigan  46104 

106.  University  of  Michigan 
Industrial  Engineering  Department 
ATTN :  Walt  Hancock 

Ann  Arbor,  Michigan  46104 

107.  Noval  Ocean  Systems  Center 
ATTN:  Dennis  McCall,  Code  6242 
San  Diego,  California  92512 

106.  Office  of  Naval  Research 

ATTN:  Marvin  Denicoff,  Code  437 
800  North  Quincy  Street 
Arlington,  Virginia  22217 

109.  Naval  Ocean  Systems  Center 
ATTN:  John  Schill,  Code  92152 
San  Diego,  California  92152 

110.  University  of  Pennsylvania 
Wharton  School  of  Business 
ATTN:  E.  Morgan 

Room  W-63,  Dietrich  Hall 
Philadelphia,  Pennsylvania  19104 


111.  Naval  Electronics  Systems  Command 
ATTN:  Dan  Schutzer,  PMI  128 

2511  Jefferson  Davis  Highway 
Washington,  D.C.  22362 

112.  Advanced  Research  Projects  Agency/IPTC 
ATTN:  R.  Kahn 

1420  Wilson  Blvd. 

Arlington,  Virginia  22229 

113.  Threshold  Technology,  Inc. 

ATTN :  Tom  Martin 

162S  Underwood  Blvd. 

Delran,  New  Jersey  06275 

114.  Threshold  Technology,  Inc. 

ATTN:  Joe  Bove 

lo29  Underwooa  Blvd. 

Delran,  New  Jersey  06075 

115.  Threshold  Technology,  Inc. 

ATTN:  Phillips  Scott 

1629  Underwood  Blvd. 

Delran,  New  Jersey  06075 

116.  Advanced  Research  Projects  Agency/CTO 
ATTN:  Craig  Fields 

1400  Wilson  Blvd. 

Arlington,  Virginia  22209 

117.  Commandant  of  the  Marines 
Scientific  Adviser 

ATTN:  A.  L.  Slafkosky,  Code  RD-1 
Washington,  D.C.  22360 

118.  Institute  for  Defense  Analysis 
ATTN:  Jesse  Crlanshy 

402  Arny-Navy  Drive 
Arlington,  Virginia  22222 

119.  Naval  Ocean  Systems  Center 
ATTN:  Glen  Allgaier,  Code  6242 
San  Diego,  California  92152 

120.  air  Force  Aerospace  Medical  Research  Lah/EIF 
ATTN:  Den  Topmiller 

Wright  Patterson  AFB,  Chio  45433 
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121.  Naval  ilectroni.cs  Systems  Command 
ATTN:  R.  Fratilla,  Code  330 

2511  Jefferson  Davis  Highway 
Arlington,  Virginia  22360 

122.  Naval  Electronics  Systems  Command 
ATTN:  J.  Machodo,  Code  330 

2511  Jefferson  Davis  Highway 
Arlington,  Virginia  22360 

123.  Capt  John  Armstrong 

6445  Sugar  Creek  Way 
Orleans,  Ontario 
Canada  A1C  1Y1 

124.  Department  of  National  Defence 
NDHS  DAS  Eng.  4 

ATTN:  Lt  Col  J.  A.  Wellington,  CAF 
121  Colonel  by  Drive 
Ottawa,  Ontario 
Canada  K1A  0E2 

125.  Rome  Air  Development  Center/IRRA 
ATTN:  R.  Vonusa 

Sriffiss  Af'B,  New  York  13441 

125.  Lt  Col  Mark  Smith,  USA? 

9520  Braddock  Road 
Fairfax,  Virginia  22032 

12?.  Computer  Corporation  of  America 
ATTN:  Chris  Eerot 
575  Technology  Square 
Cambridge,  Massachusetts  22139 

128.  Digital  Equipment  Corporation 
ATTN:  Paul  Thordarson,  ML3-2/E41 
146  Main  Street 

Maynard,  Massachusetts  02139 

129.  Thomas  J.  Watson  Researcn  Center 
ATTN :  John  Gould 

Box  218 

Yorktown  Heights,  New  York  10598 

130.  Lockheed  Missile  and  Space  Division 
Department  86-10 

ATTN:  Leon  Herman 
Box  182 
Building  504 


165 


Sunnyvale,  California  94086 

131.  Biotechnology,  Inc. 

ATTN:  Harold  Price 
3227  Rosemary  Lane 

Falls  Church,  Virginia  22042 

132.  MaJ  Warren  Watkins,  USAF 
1  STRAD/D02 

Vandenburg  AFE,  California  93437 

133.  Olin  Campbell 
Suite  201 

1150  South  State  Street 
Orem,  Utah  84057 

134.  Commander  in  Chief  Pacific  Fleet 
TRW  Field  Office 

ATTN:  George  Harris,  Code  N-34 
Pearl  Harbor,  HI  96860 

135.  Air  Force  Aerospace  Medical  Research  Lab/HEC 
ATTN:  Lew  Hahn 

Wright-Patterson  AFB ,  Ohio  45433 

136.  California  Institute  of  Technology 
Jet  Propulsion  Laboratory 
Systems  Analysis  Section 

ATTN:  Robert  L.  Sohn 
4800  Oak  Grove  Drive 
Pasadena,  California  91103 

137.  Massachusetts  Institute  of  Technology 
Lincoln  Laboratories 

ATTN:  Cliff  Weinstein 
Room  B-335 

Lexington,  Massachusetts  02173 

136.  David  Jcly 

2180  Bryant  Street 

San  Francisco,  California  94110 

139.  Naval  Training  Equipment  Center 
Human  Factors  Laboratory 
ATTN:  Elizabeth  Lambert,  Cede  N-711 
Crlando,  Florida  32813 
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140.  Threshold  Technology,  Incorporated 
ATTN:  Fred  Gladney 
Suite  4  -  Cl 

1440  South  State  College  Blvd. 

Anaheim,  California  92806 

141.  Heuristics,  Inc.  1 

ATTN:  Tom  Imperato 

1285  Hammerwood  Avenue 
Sunnyvale,  California  94266 

142.  XYEICN  1 

ATTN:  Paul  Frost 

7  Ridgedale  Avenue 

Cedar  Knolls,  New  Jersey  07927 

143.  Threshold  Technology,  Incorporated  1 

ATTN:  John  Welch 

1829  Underwood  Blvd. 

Delran,  New  Jersey  28075 

144.  Cffice  of  Naval  Research  1 

ATTN:  Robert  Wisher,  Code  458 

800  North  Quincy  Street 
Arlington,  Virginia  22217 

145.  Honeywell,  Incorporated  1 

Systems  and  Researca  Center 

ATTN:  Robert  North 
2600  Ridgeway  Blvd. 

Minneapolis,  Minnesota  55413 

146.  National  Bureau  of  Standards  1 

Information-Communications  Systems  Technology 
ATTN:  Dave  Pallett 

A219  Technology  Building 
Washington,  D.C.  22234 
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